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rinal  Technical  Report 

AFOSR  86-0182  (Adaptive  Networks) 

Dr  William  0  Berry,  Program  Manager 

Biological  Investigations  of  Adaptive  Networks: 

Neuronal  Control  of  Conditioned  Responses 

Dr  John  W  Moore  (331-30-9491),  Principal  Investigator 
Unversity  of  Massachusetts,  Amherst  01003 

I.  Summary 

Investigations  of  adaptive  neural  networks  were  conducted  using  the  classically  conditoned  nic¬ 
titating  membrane  reponse  (NMIt)  of  rabbit,  a  widely  used  model  system  for  studies  of  learning. 
Our  work  involved  both  neurobiological  and  theoretical  approaches  based  on  mathematical  models 
and  computer  simulation.  The  primary  experimental  approach  involved  recording  from  single  brain 
stem  neurons  in  awake,  behaving  animals  for  the  purpose  of  determining  the  loci  and  activity  re¬ 
lated  to  CRs.  We  developed  computational  tools  for  applying  systems  analysis  to  neurophysiological 
data  obtained  from  single-unit  recordings  from  awake  behaving  animals.  With  the  acquisition  of 
an  expanded  computer  network,  we  became  able  to  routinely  characterize  the  relationship  between 
single  neurons’  dynamic  behavior  and  the  CR  in  terms  of  differential  equations  and  sophisticated 
correlational  analyses  based  on  Fourier  and  Laplace  transform  methods.  A  second  experimental 
approach  used  WGA-HRP  for  anatomical  studies  of  involved  brain  stem/cerebellar  circuits.  This 
work  significantly  adds  to  our  understanding  of  neural  circuits  underlying  NMR  conditioning. 

Theoretical  studies  resolved  around  two  mathematical  models  of  learning.  One  is  the  Sutton- 
Barto-Desmond  (SBD)  model  which  represents  an  implementation  of  the  more  general  Sutton-Barto 
model.  The  SBD  model  was  designed  to  describe  real-time  features  of  the  NM  CR.  A  cerebellar 
network  implementation  of  this  model  was  constructed  by  combining  parametric  constraints  of 
the  model  dictated  by  behavioral  data  with  constraints  based  on  anatomy  and  physiology  of  the 
cerebellum.  The  second  major  theoretical  development  was  the  construction  of  a  two-element 
neural-network  architecture  that  elegantly  describes  adaptive  timing  as  manifested  in  the  fine-grain 
temporal  characteristics  of  CRs.  This  model,  designated  VET  in  some  of  our  papers,  overcomes 
certain  limitations  of  the  SBD  model.  A  proposed  cerebellar  implementation  of  this  model  is 
described  in  an  appended  report  and  in  forthcoming  published  articles. 

II.  Research  objectives 

The  general  objectives  for  the  reporting  period  were  the  same  as  those  stated  in  previous  reports 
going  back  to  those  submitted  in  connection  with  AFOSR  Grant  83-0215.  These  goals  and  our 
approches  to  them  remain  unchanged,  so  no  purpose  would  be  served  by  restating  them  here.  The 
status  of  work  in  the  field  and  the  contributions  of  our  laboratory  are  summarized  in  technical 
reports  listed  in  Section  IV.  I  should  mention  that  the  experimental  research  outlined  below  was 
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partially  supported  by  NSF.  This  grant  provided  some  of  the  funds  needed  for  equipment  used 
in  conjunction  with  single-unit  recording  and  anatomical  studies.  Both  AFOSR  and  NSF  are 
acknowledged  in  our  reports  because  funds  from  both  agencies  have  been  used  to  purchace  general 
purpose  equipment.  There  has  been  no  duplication  of  effort  in  connection  with  these  two  sources 
of  support. 

III.  Status  of  research 

The  main  research  efforts  over  the  past  tliree  years  are  summarized  in  an  appended  manuscript 
by  the  Principal  Investigator  entitled  "Implementing  connectionist  algorithms  for  classical  condi¬ 
tioning  in  the  brain” .  This  article  provides  an  overview  of  our  experimental  and  theoretical  efforts 
within  the  context  of  related  work  in  other  laboratories.  It  is  not  a  complete  description  of  all  of  our 
work,  however;  detailed  summaries  have  been  provided  in  prior  semi-annual  reports  and  published 
materials  (Section  IV). 

Also  appended  is  a  paper  entitled  “Single-unit  activity  in  the  rabbit  red  nucleus  during  the 
classically  conditioned  nictitating  membrane  response:  A  preliminary  report”.  This  paper  was 
included  in  this  technical  report  because,  unlike  other  single-unit  recording  studies  performed  in 
this  laboratory  during  the  past  three  years,  it  was  not  described  in  the  proposal  submitted  last 
year  which  led  to  our  current  grant,  AFOSR  89-0391. 

Finally,  this  report  lists  computer  softwareB  created  by  members  of  our  group  over  the  past 
three  years  (Section  VILA). 

IV.  Technical  reports 

Included  are  published  citations  of  work  published  and  ”in  press”  from  1986  to  the  present. 
Referencess  from  1987  were  initiated  under  AFOSR  Grant  86-0182  and  partially  supported  by  NSF 
as  explained  in  Section  II. 


1.  Moore,  J.W.,  Desmond,  J.E.,  Berthier,  N.E.,  Blazis,  D.E.J.,  Sutton,  R.S.,  Barto,  A.G.  Sim¬ 
ulation  of  the  classically  conditioned  nictitating  membrane  response  by  a  neuron-like  adap¬ 
tive  element:  Response  topography,  neuronal  firing,  and  interstimulus  intervals.  Behavioural 
Brain  Research ,  1986,  21:  143-154. 

2.  Blazis,  D.E.J.,  Desmond,  J.E.,  Moore,  J.W.,  Berthier,  N.E.,  Sutton,  R.S,  and  Barto,  A.G. 
Simulation  of  the  classically  conditioned  nictitating  membrane  response  by  a  neuron-like 
adaptive  element:  A  real-time  variant  of  the  Sutton-Barto  model.  Proceedings  of  the  Eighth 
Annual  Conference  of  The  Cognitive  Science  Society.  Hillsdale,  N.  J.:  Lawrence  Erlbaum 
Associates,  1986,  176-186. 

3.  Schmajuk,  N.A.  and  Moore,  J.W.  A  real-time  attentional-associative  network  for  classical 
conditioning  of  the  rabbit’s  NMR.  Proceedings  of  the  Eighth  Annual  Conference  of  The  Cog¬ 
nitive  Science  Society.  Hillsdale,  N.  J.:  Lawrence  Erlbaum  Associates,  1986,  794-807. 


2 


4.  Desmond,  J.E.,  Blazis,  D.E.J.,  and  Moore,  J.W.  Computer  simulations  of  a  classically  con¬ 
ditioned  response  using  neuron-like  adaptive  elements:  Response  topography.  Society  for 
Neuroscience  Abstracts,  1986,  12:  516. 

5.  Rosenfield,  M.E.  and  Moore,  J.W.  HRP- WG  A  studies  of  premotor  cerebellar-brain  stem  path¬ 
ways  for  the  classically  conditioned  nictitating  membrane  response.  Society  for  Neuroscience 
Abstracts,  1986,  12:  752. 

6.  Berthier,  N.E.  and  Moore,  J.W.  Cerebellar  Purkinje  cell  activity  related  to  the  classically 
conditioned  nictitating  membrane  response.  Society  for  Neuroscience  Abstracts,  1986,  12: 
1418. 

7.  Berthier,  N.E.  and  Moore,  J.W.  Cerebellar  Purkinje  cell  activity  related  to  the  classically 
conditioned  nictitating  membrane  response.  Experimental  Brain  Research,  1986,  63:  341- 
350. 

8.  Desmond,  J.  E.  and  Moore,  J.  W.  Dorsolateral  pontine  tegmentum  and  the  classically  con¬ 
ditioned  nictitating  membrane  response:  Analysis  of  CR-related  single-unit  activity.  Experi¬ 
mental  Brain  Research,  1986,  65:  59-74. 

9.  Berthier,  N.E.,  Desmond,  J.E.,  and  Moore,  J.W.  Brain  stem  control  of  the  nictitating  mem¬ 
brane  response.  In  Gormezano,  I.,  Prokasy,  W.F.,  and  Thompson,  R.F.  (Eds.),  Classical 
Conditioning,  3rd  Edition.  Hillsdale,  NJ:  Lawrence  Erlbaum  Associates,  1987,  275-286. 

10.  Schmajuk,  N.A.  and  Moore,  J.W.  Two  Attentional  Models  of  Classical  Conditioning:  Varia¬ 
tions  in  CS  Effectiveness  Revisited.  University  of  Massachusetts  at  Amherst,  Department  of 
Computer  and  Information  Science,  Technical  Report  87-29,  1987,  33  pages. 

11.  Blazis,  D.E.J.  and  Moore,  J.W.  Simulation  of  a  Classically  Conditioned  Response:  Com¬ 
ponents  of  the  Input  Trace  and  a  Cerebellar  Implementation  of  the  Sutton-Barto-Desmond 
Model.  University  of  Massachusetts  at  Amherst,  Department  of  Computer  and  Information 
Science,  Technical  Report  87-74,  1987,  61  pages. 

12.  Moore,  J.W.  and  Berthier,  N.E.  Purkinje  cell  ac  ,vity  and  the  conditioned  nictitating  mem¬ 
brane  response.  In  Glickstein,  M.,  Yeo  C.,  and  .’,  J  (Eds.),  Cerebellum  and  Neuronal 
Plasticity,  New  York:  Plenum,  1987,  339-352. 

13.  Desmond,  J.E.  and  Moore,  J.W.  Red  nucleus  single-unit  activity  during  the  classically  con¬ 
ditioned  rabbit  nictitating  membrane  response.  Society  for  Neuroscience  Abstracts,  1987,  13: 
841. 

14.  Blazis,  D.E.J.  and  Moore,  J.W.  A  cerebellar  cortical  implementation  of  the  Sutton-Barto- 
Dtomond  model  of  the  classically  conditioned  rabbit  nictitating  membrane  response.  Society 
for  Neuroscience  Abstracts,  1987,  13:  842. 

15.  Desmond,  J.E.  and  Moore,  J.W.  Adaptive  timing  in  neural  networks;  The  conditioned  re¬ 
sponse.  Biological  Cybernetics,  1988,  58:  405-415. 


3 


16.  Desmond,  J.E.  Temporally  adaptive  conditioned  responses:  Representation  of  the  stimulus 
trace  in  neural-network  models.  University  of  Massachusetts  at  Amherst,  Department  of 
Computer  and  Information  Science,  Technical  Report  88-80,  1988,  52  pages. 

17.  Schmajuk,  N.A.  and  Moore,  J.W.  The  hippocampus  and  the  classically  conditioned  nictitating 
membrane  response:  A  real-time  attentional-associative  model.  Psychobiology,  1988,  16:  20- 
35. 

18.  Berthier,  N.E.,  Barto,  A.G.,  and  Moore,  J.W.  Linear  systems  analysis  of  cerebellar  deep 
nuclei  cells  during  performance  of  classically  conditioned  eyeblink.  Society  for  Neuroscience 
Abstracts,  1988,  14:  1239. 

19.  Rosenfield,  M.E.  and  Moore,  J.W.  Is  there  a  reciprocal  connection  between  red  nucleus  and 
interposed  cerebellar  nuclei  in  rabbit?  Society  for  Neuroscience  Absracts,  1988,  14:  493. 

20.  Moore,  J.W.  and  Blazis,  D.E.J.  Cerebellar  Implementation  of  a  Computational  Model  of 
Classical  Conditioning.  In  Strata,  P.  (Ed.),  The  olivocerebellar  system  in  motor  control. 
Berlin:  Springer- Verlag,  1989,  387-399. 

21.  Moore,  J.W.  and  Blazis,  D.E.J.  Simulation  of  a  classically  conditioned  response:  a  cerebellar 
neural  network  implementation  of  the  Sutton-Barto-Desmond  model.  In  Byrne,  J.  H.  and 
Berry,  W.  0.  (Eds.),  Neural  models  of  plasticity.  Experimental  and  theoretical  approaches. 
New  York:  Academic  Press,  1989,  187-207. 

22.  Moore,  J.W.  and  Blazis,  D.E.J.  Conditioning  and  the  cerebellum.  In  Arbib,  M.A.  and  Amari, 
S.  Dynamic  interactions  in  neural  networks:  Models  and  networks.  Berlin:  Springer- Verlag, 
1989,  261-277. 

23.  Schmajuk,  N.A.  and  Moore,  J.W.  Effects  of  hippocampal  manipulations  on  the  classically 
conditioned  ni"titating  membrane  response:  simulations  by  an  attentional-associative  model. 
Behavioural  Brain  Research,  1989,  32:  173-189. 

24.  Moore,  J.W.  Cerebro-cerebellar  learning  loops  and  language.  Peer  commentary,  Behavioral 
and  Brain  Sciences,  1989,  12:  156. 

25.  Moore,  J.W.  and  Desmond,  J.E.  A  cerebellar  neural  network  implementation  of  a  temporally 
adaptive  conditioned  response.  In  Gormezano,  I.  (Ed.),  Learning  and  memory:  The  biological 
substrates.  Hillsdaie,  NJ:  Lawrence  Erlbaum  Associates.  In  press. 

26.  Coulter,  D.A.,  Lo  Turco,  J.J.,  Kubota,  M.,  Disterhoft,  J.F.,  Moore,  J.W.,  and  Alkon,  D.L. 
Classical  conditioning  reduces  the  amplitude  and  duration  of  the  calcium- dependent  afterhy¬ 
perpolarization  in  rabbit  pyramidal  cells.  Journal  of  Neurophysiology,  1989,  61:  971-981. 

27.  Moore,  J.W.,  Desmond,  J.E.  and  Berthier,  N.E.  Adaptively  timed  conditioned  responses  and 
the  cerebellum:  A  neural  network  approach.  Biological  Cybernetics.  In  press. 
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28.  Moore,  J.W.  Implementing  connectionist  algorithms  for  classical  conditioning  in  the  brain. 
In  Commons,  M.,  Grossberg,  S.,  and  Staddon,  J.E.R.  (Eds.),  Neural  network  models  of  con¬ 
ditioning  and  action.  Hillsdale,  N.J.:  Lawrence  Erlbaum  Associates.  In  press. 

29.  Moore,  J.W.,  Berthier,  N.E.,  and  Blazis,  D.E.J.  Classical  eye  blink  conditioning:  Brain  sys¬ 
tems  and  implementation  of  a  computational  model.  In  Gabriel,  M  and  Moore,  J.W.  (Eds.), 
Learning  and  computational  neuroscience.  Cambridge,  MA:  MIT  Press.  In  press. 

V.  Professional  personnel 

1.  John  W  Moore,  Ph  D,  Psychology  (Experimental/Learning),  Indiana  University,  1962.  Pro¬ 
fessor  of  Psychology  (Biopsychology);  Core  Faculty,  Neurosciece  and  Behavior  (MSB)  Program  ; 
Associated  Professor  of  Computer  and  Information  Science. 

2.  Neil  E  Berthier,  Ph  D,  Psychology  (Neuroscience  and  Behavior),  University  of  Massachusetts, 
1981.  Senior  Research  Associate.  Dr  Berthier  serves  as  a  resource  person  and  colleague.  His  salary 
has  been  provided  by  NSF  Grant  BNS  85-06787,  on  which  he  is  co-PI. 

3.  John  E  Desmond,  Ph  D,  Psychology  (Neuroscience  and  Behavior),  University  of  Mas¬ 
sachusetts,  1985.  Research  Associate. 

4.  William  G  Richards,  Ph  D,  Pychology  (Neuroscience),  Princeton  University,  1985.  Research 
Associate. 

5.  Marcy  E  Rosenfield,  B  S  (Zoology),  University  of  Massachusetts,  1982.  Rosenfield’s  formal 
title  is  Departmental  Assistant.  She  is  a  certified  AALAS  animal  care  technician. 

VI.  Interactions 

Unless  otherwise  specified,  the  professional  interactions  listed  in  this  section  refer  to  the  PI. 
The  material  is  broken  down  by  year.  The  listings  from  Years  One  and  Two  are  taken  verbatim 
from  previous  annual  technical  reports. 

Year  One: 

1.  Slide  session  chairperson  and  member  of  the  program  committee,  Eighth  Annual  Conference 
of  the  Cognitive  Science  Society,  University  of  Massachusetts  -  Amherst,  August,  1986. 

2.  Poster  presentations,  etc.,  Society  for  Neuroscience  meetings,  Washington,  D  C,  November, 
1986. 

3.  Participant  in  plenary  session  on  Cerebellum  and  Learning  (with  R  F  Thompson,  W  Welker, 
and  S  G  Lisberger),  Winter  Conference  on  Neurobiology  of  Learning  and  Memory,  Park  City,  Utah, 
January  1987. 

4.  Speaker  at  Conference  on  Neural  Models  of  Plasticity,  Woods  Holes  MA,  April-May,  1987. 
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5.  Speaker  and  Participant  at  U  S  -  Japan  Seminar  on  Competition  and  Cooperation  in  Neural 
Nets  2,  University  of  Southern  California,  May,  1987. 

Other  principal  interactions  involve  ongoing  interactions  (e  g,  classes  and  seminars)  and  collab¬ 
orative  relationships  with  colleagues:  A  G  Barto,  R  S  Sutton,  M  Jordan,  A  H  Klopf,  E  J  Kehoe, 

N  A  Schmajuk,  J  Ayres,  and  others. 

Because  it  involved  grant  supported  travel,  I  should  specifically  note  that  J  E  Desmond,  D 
E  J  Blazis  (graduate  student),  and  I  visited  A  H  Klopf  at  Wright  Aeronautical  Laboratories  for 
two  days  in  June,  1986  to  conduct  simulation  studies  of  Klopf ’s  drive-reinforcement  model  and  R 
S  Sutton  and  A  G  Barto’s  time-difference  model  (formerly  known  as  the  adaptic  heuristic  critic 
element). 

Year  Two: 

1.  Principal  Participant:  Satellite  Symposium  of  the  2nd  IBRO  World  Congress  of  Neuroscience, 
The  Olivocerebellar  System  and  Motor  Control ,  Turin,  Italy,  August,  1987. 

2.  Invited  colloquia  and  seminars  during  1987-88  academic  year:  Wesleyan  University,  Univer¬ 
sity  of  Texas,  University  of  Illinois. 

3.  Presentations  at  meetings:  AFOSR  Life  Science  Program  Review,  Brooks  AFB,  Texas, 
December,  1987;  Annual  Winter  Conference  on  Neurobiology  of  Learning  and  Memory,  Park  City, 
Utah,  January,  1988;  Invited  lecture,  Midwestern  Psychological  Association  meetings  in  Chicago, 
April,  1988. 

4.  Grant  Panels  Activity:  NIH  Biopsychology  Panel  (ad  hoc),  October,  1987.  This  included  site 
visits  to  UCIrvine  and  UCLA  during  September,  1987;  NIMH  Behavioral  Neurobiology  Subcom- 
mitee  (ad  hoc),  October,  1987;  NSF  panel  on  Research  Experiences  for  Undergraduates  (Psychobi¬ 
ology  sub  panel),  February,  1988;  ONR  program  review  in  Cognitive  and  Neural  Science  (Board  of 
Visitors),  February,  1988. 

5.  Reviewing  and  Related  Activity:  Consulting  editor  for  Psychobiology  and  reviewed  manuscripts 
for  Journal  of  Experimental  Psychology:  Animal  Behavior  Processes,  Psychological  Bulletin,  Behav¬ 
ioral  Brain  Research,  MIT  Press.  Member  of  Advisory  Board  for  a  new  monograph  series,  Research 
Notes  in  Neural  Computing,  to  be  published  by  Springer- Verlag. 

6.  I  was  honored  in  being  asked  to  nominate  possible  recipients  of  this  year’s  Kyoto  Prize  (a 
Nobel  class  monetary  award  given  by  Japan’s  Inamori  Foundation)  for  work  in  Cognitive  Science. 
The  prize  was  ultimately  awarded  to  Noam  Chomsky. 

7.  Continuing  strong  interactive  relationships  with  A  G  Barto  and  other  colleagues  working 
in  Adaptive  Networks:  A  H  Klopf’s  group  at  Wright-Patterson,  R  S  Sutton  at  GTE  Labs,  N  A 
Schmajuk  presently  at  Northwestern  University  and  others. 
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8.  Our  relationship  with  Barto’s  group  continues  to  strengthen  our  relationship  with  the  uni¬ 
versity’s  Department  of  Computer  and  Information  Sciences  where  I  am  Associated  Professor. 

9.  Drs  Berthier  and  Desmond  became  Associated  Faculty  of  the  university’s  NSB  Program. 
They  have  also  been  asked  to  review  grant  proposals  and  submitted  manuscripts. 

Year  Three: 

1.  Perhaps  the  most  significant  interactions  occurred  as  part  of  team- taught  graduate  level 
seminar  offered  by  my  colleague  in  computer  science  and  myself  (COINS  891  A,  fall,  1989,  3  credits). 
The  genernal  topic  was  behavioral,  biological,  and  computational  approaches  to  learning. 

2.  The  fall  of  1988  also  provided  opportunities  for  substantial  interaction  with  Dr  James  Houk, 
a  cerebellar  and  motor  system  physiologist  who  is  chair  of  the  physiology  department  at  Northwest¬ 
ern  University  Medical  School.  Dr  Houk  spent  part  of  his  sabbatical  semester  here,  and  interactions 
with  him  were  mutually  rewarding  in  two  main  areas:  (a)  quantitative  treatment  of  neural  neural 
data  related  to  movement — especially  systems  analysis;  (b)  cerebellar  implementation  of  computa¬ 
tional  models. 

3.  I  gave  a  major  invited  presentation  at  a  satellite  symposium  organized  by  D.  Alkon  and  C. 
Woody  held  in  connection  with  the  1988  meetings  of  the  Society  for  Neuroscience  in  Toronto. 

4.  I  gave  an  invited  talk  at  the  annual  Winter  Conference  on  the  Neurobiology  of  Learning  and 
Memory  convened  at  Park  City,  Utah,  January,  1989. 

5.  I  gave  an  invited  talk  to  the  neuroscience  group  at  Oxford  University  in  March,  1989. 

6.  I  gave  an  invited  lecture  at  the  Twelfth  Symposium  on  Models  of  Behavioral  convened  at 
Harvard  University,  June,  1989. 

7.  Neil  Berthier  gave  an  invited  talk  in  the  department  of  neurobiology  at  SUNY  Stony  Brook, 
February,  1989. 

8.  John  Desmond  gave  an  invited  talk  to  the  psychology  department  of  Brown  University  in 
March,  1989. 

9.  Diana  Blazis  (graduate  student  now  working  on  her  dissertation)  gave  an  invited  talk  to  the 
psychobiology  group  of  Yale  University,  May,  1989. 

VII.  New  discoveries 

Discoveries  that  might  be  designated  as  new  were  those  stemming  from  experimental  research, 
e.g.,  the  anatomical  work  discussed  above  in  published  and  forthcoming  technical  reports. 
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VIIA.  Software  developed 

Most  of  the  software  developed  over  the  past  three  years  was  designed  to  run  on  our  local 
network  of  Sun  workstations  (SUN  LAN),  which  includes  a  DEC  LN03-plus  laser  printer,  dumb 
terminals,  and  interfaces  for  Apple  lie  and  other  small  systems.  Summaries  of  software  written  by 
Neil  Berthier,  Diana  Blazis,  and  Bill  Richards  appear  below. 

Additional  software  has  been  written  by  John  Desmond.  Because  of  its  length,  Desmond’s 
summary  appears  as  a  separate  appendix  to  this  report. 

Software  by  Neil  Berthier: 

Dr  Berthier  has  done  most  of  the  system  programming,  and  he  has  created  several  programs 
for  performing  sophisticated  analyses  of  neural  and  behavioral  data. 

•  Upgrade  SunOS  software  as  it  became  available,  applied  patches  as  necessary. 

•  Install  X  window  system,  with  Purdue  patches.  Install  and  debug  contributed  software  in¬ 
cluding:  x-graph-11,  texx2.9,  xlock,  xviewsun,  xgdb,  twm,  xdbx,  xfig. 

•  Install  and  debug  text  formating  and  printing  software  based  on  TeX,  LaTeX,  BibTex,  SliTex, 
dvi21n03,  ln03dvi,  dvitool.  Write  LaTeX  macros. 

•  Install  Gnu  software,  gnu-gcc,  gnu-gdb,  gnu-emacs.  Write  macros  for  gnu-emacs. 

•  Write  a  graphics  driver  for  DEC  ln03-plus. 

•  Write  a  system  of  programs  to  do  basic  analysis  of  spike  data. 

-  periresponse.c  -  compute  periresponse  histograms. 

-  mannwhn.c  -  test  null  hypothesis  about  firing  patterns  using  a  Mann- Whitney  test. 

-  kolsmir.c  -  test  null  hypothesis  about  firing  patterns  using  a  Kolmogorov- Smirnow  test. 

-  binom.c  -  test  null  hypothesis  about  firing  patterns  using  a  Binomial  test. 

-  prelim.c  -  compute  basic  parameters  of  cell  firing  and  test  null  hypothesis  about  firing 
patterns  using  a  poisson  test. 

-  gamm.c  -  test  null  hypothesis  about  firing  patterns  using  a  gamma  distribution  hypoth¬ 
esis. 

•  A  system  of  programs  to  compute  digital  filter  coefficients,  and  to  digitally  filter  data. 

-  kaiser. c  -  compute  kaiser  filter  coefficients. 

-  marr.c  -  compute  Marr-Hildreath  filte  coefficients. 

-  filter. c  -  filter  spike  data  with  digital  filter. 

-  filterdis.c  -  display  filtered  data  using  SunView. 

-  perifd.c  -  display  periresponse  filtered  data. 
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•  A  system  of  programs  to  do  fast  Fourier  transforms  (FFTs). 

-  fft.c  -  do  foward  FFT. 

-  ifft.c  -  do  inverse  FFT. 

-  fdat.c  -  do  FFT  of  data. 

-  fftdis.c  -  display  FFT  of  data. 

-  fcoef.c  -  do  FFT  of  filter  coefficients. 

•  A  system  of  programs  to  do  linear  systems  analyses  of  data. 

-  recur. c  -  a  program  that  computes  time  series  analysis  coefficients  by  inverting  matrices. 

-  convolv.c  -  use  periodic  convolution  to  get  predicted  output  of  the  linear  system. 

-  mserr.c  -  compute  multiple  R. 

-  correlation. c  -  use  FFT  and  iFFT  to  do  cross-correlational  analysis. 

-  disp-correlation.c  -  display  results  of  cross-correlation  with  SunView. 

-  transf.c  -  compute  transfer  function  for  a  set  of  coefficients. 

-  transdislog. c  -  display  and  fit  transfer  functions  using  SunView. 

•  A  system  of  programs  to  compute  time  series  coefficients  using  QR  decomposition. 

-  recurjnain.c  -  main  program  to  compute  coefficients. 

-  QRdecomp.c  -  programs  based  on  LINPACK  to  do  QR  decomposition  using  Householder 
transformations. 

-  float-signal. c  -  catches  floating  point  errors. 

•  Many  Unix  scripts  to  resolve  systems  problems. 

Software  by  Diana  Blazis: 


•  General  Simulator.  Computes  inputs,  outputs  and  update  rules  for  models  of  adaptive 
behavior.  The  program  can  be  run  interactively  or  automatically  (“batch”  processing).  In 
interactive  mode  the  program  provides  popup  menu.,  which  allow  the  user  to  change  learning 
algorithms,  model  parameters,  and  learning  procedures  at  will.  The  program  displays  the 
appearance  of  the  paradigm,  computes  the  experiment,  and  plots  model  output  in  several 
forms.  In  batch  mode,  the  program  reads  prepared  command  files  and  automatically  gen¬ 
erates  hardcopy  graphics  output  on  request.  Presently,  the  simulator  runs  several  learning 
algorithms  for  many  different  stimuli.  The  modular  form  of  the  simulation  program  allows 
for  future  incorporation  of  other  learning  rules.  Graphics  output,  which  includes  response 
waveforms  for  acquisition  and  terminal  trials,  learning  curves,  and  spike  histograms,  can  be 
enabled  or  disabled  at  operator  request. 
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•  nmrtool.  Interactive  program  for  analysis  of  data  obtained  during  classical  conditioning 
trials.  Data  collected  using  our  Apple  First  system  are  written  to  disk  and  then  uploaded 
to  the  SUN  LAN.  nmrtool  displays  responses  for  all  trials.  Several  response  measures  are 
computed,  including  peak  and  onset  latencies,  CR  amplitudes,  and  CR  areas.  Learning  curves 
are  plotted.  Hardcopy  of  nmrtool  results  is  available  at  user  request,  nmrtool  generates 
summary  files  that  used  for  automated  statistical  analysis  of  the  data.  Future  plans  for 
nmrtool  include  a  user  interface  to  program  analysis  of  data  obtained  during  any  classical 
conditioning  procedure. 

•  Related  projects,  (a)  Basic  nmrtool:  used  prior  to  the  implementation  of  nmrtool  for 
the  SUN  LAN.  Data  from  behavioral  experiments  were  analyzed  on  our  Apple  He’s  using 
a  modifified  version  of  an  analysis  program  prepared  by  John  Desmond  and  Bill  Richards, 
(b)  File  conversion  routines:  data  obtained  from  the  FIRST  system  that  controls  behavioral 
experiments  via  Apple  lies  and  collects  data  must  be  converted  for  transfer  to  the  SUN 
LAN.  File  conversion  is  coded  in  Basic  and  executed  on  the  Apple  lie.  (c)  Applications 
programming  and  maintenance  for  our  FIRST  system,  e.g.  programming  stimulus  control  for 
new  experiments,  (d)  Assorted  small  programs  for  statistical  manipulation  of  output  files  of 
nmrtool. 


Software  by  Bill  Richards: 

Dr  Richards  wrote  software  that  enables  a  Commodore  C-64  computer  to  control  physiological 
recording  experiments.  This  software  is  readily  transferable  to  other  computers  using  the  6502- 
series  8-bit  processor. 


•  Acq/ctrl.  This  program  is  a  B ASIC/assembly  language  hybrid  that  controls  CS  and  US 
presentations  for  differential  conditioning,  detects  CRs,  counts  neural  spikes  in  two  separate 
channels,  and  provides  on-line  cumulative  histograms  of  spike  activity  compiled  separately 
for  CS+  and  CS-  trials  and  for  CR  and  non-CR  trials. 

•  EVENT.  Assembly  language  program  for  off-line  trial-by-trial  analysis  of  event  timing. 
Events  on  eight  channels  are  timed  with  0.1  msec  resolution.  The  program  also  controls 
A/D  sampling  of  voltage  trace  from  NMR  transducer  and  disk  files  for  NMR,  neural  spikes, 
and  synchronization. 

•  HSAD.  Basic/assembly  language  program  which  controls  high-speed  digitization  of  a  single 
channel  of  neural  spike  activity  for  (a)  confirmation  of  window  discrimination  (reads  and  edits 
corresponding  disk  files  generated  by  EVENT)  or  (b)  graphic  plotting. 

•  FORMAT  Converts  EVENT  flies  downloaded  from  C-64  to  format  which  ANALYZE  (by 
John  Desmond)  can  read. 

•  FIG. HIRES  Reads  C-64  HSAD  files  and  EVENT  NMR  files  and  generates  oscilloscope-like 
figures  showing  neural  activity  and  NMR  for  single  trial  on  laser  printer  (LN03).  An  example 
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is  shown  in  Figure  1.  Previously,  such  illustrations  required  costly  and  time  consuming 
photograph ’c  processing. 


John  E.  Desmond  -  software  development 


C/Unix  software  for  the  Sun 

iball  is  a  mouse-driven  SunView  program  that  allows  user  to  examine  individual  trials 
from  a  recording  session.  The  CR,  spikes,  cumulative  sum  of  spikes,  and  the  cu¬ 
mulative  sum  adjusted  for  baseline  firing  is  displayed.  By  using  the  mouse,  the  CR 
onset,  CR  duration,  CR  velocity,  burst  onset,  burst  duration,  burst  velocity,  and 
burst  depth  of  modulation  is  computed  and  displayed.  In  addition,  CR  area  and 
maximum  CR  amplitude  are  automatically  computed.  The  data  can  be  stored  for 
further  analyses  and  subsequently  reloaded  for  modifications.  The  user  also  has  op¬ 
tion  of  displaying  the  following  averaged  plots:  (a)  CS+  or  CS-  trials  (b)  CR  trials 
(c)  NO  CR  trials,  (d)  CR  trials  adjusted  for  CR  onset.  Sample  output  is  depicted  in 
Figures  1,  2,  and  3. 

binom  performs  binomial  statistical  test  on  CR  and  NO  CR  trials.  Tests  the  null  hy¬ 
pothesis  that  firing  rate  is  identical  for  the  two  trial  types  within  a  specified  time 
interval,  as  described  by  Dorrscheidt  (1981). 

plot  Jsi  outputs  interspike  interval  histograms  of  cell  baseline  firing.  The  output  is  stored 
in  a  file  in  a  way  that  can  be  easily  searched  with  standard  Unix  search  functions. 

cusums  generates  the  following  graphical  display  (SunView)  for  5  cells  at  a  time:  (a) 
averaged  P3TH  for  CR  trials,  (b)  averaged  PSTH  for  NO  CR  trials,  and  (c)  averaged 
PETH  for  CR  trials.  The  number  of  trials  for  each  graph  is  also  displayed.  The  user 
has  the  option  to  include  or  remove  time  calibration  grid.  The  program  can  also  be 
set  up  in  an  autoprint  mode  for  batch  printouts  of  all  cells. 

ave-changepoint  (in  collaboration  with  Neil  Berthier)  displays  averaged  filtered  neuronal 
activity  for  CR  and  Non-CR  trials.  Filtered  waveforms  are  displayed  in  a  peristimulus 
plot  (synced  to  the  CS  onset)  and  in  a  perievent  plot  (synced  to  the  CR  onset).  Nine 
cells  ri  -.lsplayed  per  screen,  and  screens  can  be  printed  out  as  batch  job  in  autoprint 
moo-  'he  purpose  of  the  software  is  to  find  the  time(s)  at  which  neuronal  firing 
rate  charges.  These  times  can  be  measured  from  the  zero-crossing  points  of  the 
wavefor. 

simdata  simulates  CRs  and  spikes.  The  program  is  designed  to  simulate  neuronal  ac¬ 
tivity  that  is  stimulus-dependent,  movement-dependent,  or  both.  Activity  that  is 
movement-dependent  can  respond  to  the  position,  velocity,  or  acceleration  of  the 
movement,  and  can  either  lead  or  lag  the  simulated  CR.  Stimulus-dependent  activity 
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occurs  at  a  fixed  latency  relative  to  the  onset  of  the  CS.  The  purpose  of  this  program 
is  to  generate  data  with  known  input-output  properties  for  comparison  with  genuine 
data. 

classify  allows  user  to  place  cell  numbers  into  different  category  files.  The  program  is 
basically  designed  for  data  organization. 

plot-cells  is  a  mouse-driven  CGI  program  for  digitizing  recording  site  locations.  An 
anatomical  atlas  is  drawn  on  the  screen.  The  user  then  marks  the  recording  location 
by  pressing  a  mouse  button,  and  the  coordinates  of  the  recording  site  are  stored. 

display  _cells  displays  specified  recording  sites  on  screen  atlas,  using  the  coordinates  gen¬ 
erated  in  the  “plot_cells”  program.  Sample  output  is  depicted  in  Figure  4. 

bib  is  a  reference  collection  and  formatting  program.  The  program  extracts  references 
from  a  document  and  then  generates  a  bibliography  in  the  journal’s  preferred  format. 

findref  is  a  database  search  program.  The  user  enters  key  words  and  the  program  returns 
references  that  contain  those  keys. 

raz  is  a  SunView  program  for  manipulating  raster  files  on  a  simulated  sheet  of  8.5  X  11.0 
inch  paper.  The  user  can  adjust  the  position  of  multiple  raster  images  and  create 
a  composite  picture.  The  simulated  sheet  of  paper  can  be  rotated,  and  the  images 
can  be  magnified  or  reduced.  When  the  desired  composite  picture  is  achieved,  a 
“snapshot”  of  the  picture  can  be  taken  and  printed  on  a  LN03  laser  printer.  Sample 
output  is  depicted  in  Figure  5. 

mmolar  is  a  neural  network  model  of  classically  conditioned  responses.  The  program 
generates  graphical  (CGI)  display  of  CR  topography  and  synaptic  weights.  (See 
Desmond,  1988;  Desmond,  in  press;  Desmond  and  Moore,  1988;  Moore,  Desmond, 
and  Berthier,  in  press). 

fanout  simulates  spreading  activation  in  a  planar  array  of  neuron-like  elements.  See  pp. 
33-43  in  Desmond  (1988). 

nmr  (in  collaboration  with  Diana  Blazis)  is  a  SunView  application  for  displaying  and 
analyzing  nictitating  membrane  response  behavioral  data  collected  on  the  APPLE 
FIRST  system. 
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BASIC/ Assembly  Language  software  for  Apple  lie 

makefiles  converts  FIRST-formatted  data  into  binary  files  that  are  suitaDle  for  uploading 
to  the  Sun. 

spktime  is  an  interrupt-driven  assembly-language  program  that  reads  information  from 
3  tracks  of  a  magnetic  (VCR)  tape.  The  first  track  contains  analog  (behavioral) 
information.  The  program  digitizes  this  information  at  5  ms  intervals.  The  second 
track  consists  of  neuronal  events  (action  potentials).  The  program  records  the  times 
at  which  the  action  potentials  occur  to  250  microsec  accuracy.  The  program  decodes 
trial  type  information  and  synchronous  pulses  from  a  third  track.  Data  are  stored 
on  floppy  disks  and  uploaded  to  a  Sun  workstation  for  subsequent  analyses.  The 
program  queries  the  user  for  the  starting  and  ending  values  of  the  tape  counter  for 
each  cell,  and  stores  this  information  in  a  file;  thus,  the  experimenter  obtains  a 
comprehensive  record  of  taped  data  that  facilitates  later  retrieval. 

Pavlov  is  a  B ASIC/assembly  language  program  for  running  classical  conditioning  experi¬ 
ments  on  the  Apple  He  while  performing  neurophysiological  recordings.  The  program 
presents  various  conditioned  and  unconditioned  stimuli  to  the  subject,  and  codes  each 
trial  type  presented  on  one  track  of  a  VCR  magnetic  tape.  The  trial  types  that  are 
presented  to  the  subject  can  be  changed  during  the  session  with  the  press  of  a  key. 
The  number  and  types  of  trials  presented  are  displayed  on  the  monitor.  Subject 
numbers  and  training  protocols  are  stored  to  minimize  errors  in  training.  The  pro¬ 
gram  has  a  “collision  mode”  for  perfoming  neurophysiological  collision  experiments. 
In  this  mode,  the  computer  is  interfaced  with  a  window  discriminator.  TTL  out¬ 
put  from  the  discriminator  (caused  by  a  spontaneously  occurring  action  potential) 
is  detected  and  this  output  triggers  an  oscilloscope  sweep  and  initiates  a  precisely 
timed  delay-interval  (delay  controlled  by  the  experimenter).  At  the  end  of  the  de¬ 
lay,  the  computer  triggers  2  stimulation  pulses.  If  only  the  second  of  these  pulses 
evokes  an  antidromic  response,  then  the  first  stimulation  pulse  has  collided  with  the 
spontaneously  occurring  spike. 


I  modified  the  Apple  version  of  the  public  domain  software  “kermit”  so  that  an  entire 
disk’s  data  can  be  uploaded,  rather  than  one  file  at  a  time.  This  significantly  reduces  the 
amount  of  time  that  the  user  must  devote  to  uploading. 
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Figure  1:  A  single  trial  displayed  in  the  iball  software.  The  traces  dsiplayed  are,  from  top  to 
bottom,  digitized  behavioral  response,  spike  train,  cusum,  and  cumulative  spike  counts.  The  two 
vertical  lines  represent,  from  left  to  right,  CS  onset  and  US  onset.  The  straight  line  fitted  on  the 
cusum  graph  was  drawn  by  the  user.  From  the  slope  of  this  line,  the  software  computes  the  depth 
of  modulation  of  the  neuronal  burst.  On  this  trial,  the  neuronal  burst  is  firing  at  113.8  spikes/sec 
above  the  baseline  firing  rate. 


5 


Averaged  Plot:  CR  PERI-EVENT.  Number  of  trials:  SO.  Mean  CR  onset  -  588  ms 
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Figure  2:  Output  from  the  iball  software.  Depicted  is  behavior,  cusum,  and  cumulative  spikes 
averaged  over  all  CR  trials.  Traces  are  synchronized  to  the  CR  onset  (vertical  line).  From  this 
plot,  the  user  can  estimate  temporal  relationship  between  neuronal  activity  and  the  behavior.  In 
this  example,  neuronal  firing  precedes  the  behavior  by  60  ms. 
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Figure  3:  The  raster  mode  of  the  iball  software  allows  user  to  take  advantage  of  iball’s  time 
measurement  functions  on  graphs  generated  from  other  programs.  In  this  example,  the  user  has 
measured  a  CS-evoked  burst  of  activity  beginning  at  392  ms  and  lasting  for  74  ms. 


Figure  4:  Output  from  the  display -cells  software.  Coordinates  of  all  cells,  which  have  been 
obtained  with  the  plot-cells  software,  are  retrieved  and  cell  positions  are  designated  by  asterisks. 
The  user  can  display  any  subset  of  the  cell  population  by  entering  those  cell  numbers.  The  program 
also  has  an  extraction  function.  With  this  function  the  user  specifies  a  region  on  the  screen  and 
the  numbers  of  all  cells  that  are  located  within  the  region  are  printed  out. 
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Figure  5:  An  example  of  the  raz  software.  This  figure  illustrates  5  raster  images  on  the  screen. 
The  screen  can  be  scrolled  up/down  and  right/left  using  the  scrollbars.  In  the  center  of  the  display 
is  a  miniature  view  of  the  positions  of  the  raster  images  as  they  would  appear  on  an  8.5  in.  by  11.0 
in.  sheet  of  paper.  (This  miniature  view  can  be  moved  or  hidden  by  the  user).  The  images  can 
be  enlarged  or  reduced,  and  the  page  can  be  rotated.  The  user  can  select  an  image  and  change  its 
position  by  moving  the  mouse  cursor  and  pressing  a  button.  When  the  user  is  satisfied  with  the 
positions  of  the  images,  a  “snapshot”  of  the  composite  picture  can  be  made  and  sent  to  the  printer. 
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Implementing  Connectionist  Algorithms  for  Classical  Conditioning  in  the  Brain 

Abstract  Simple  connectionist  models  of  learning  that  conform  to  the  Widrow-Hoff  rule  can 
be  parameterized  and  extended  to  describe  real-time  features  of  classical  conditioning.  These 
features  include  the  dependence  of  learning  on  the  moment-to-moment  status  of  input  to  the 
computational  system  and  on  the  desired  topography  of  its  output.  Using  the  classically 
conditioned  nictitating  response  (NMR)  of  the  rabbit  as  a  prototypal  system,  my  coworkers 
and  I  have  devised  models  which  successfully  meet  these  real-time  criteria.  Two  models  and 
neural  network  architectures  are  described.  The  first  consists  of  a  single  neuron-like  processor 
with  learning  rules  based  on  the  Sutton-Barto  model.  The  second  consists  of  two  neuron-like 
units  with  input  based  on  a  tapped-delay  line  representation  of  stimuli.  Using  anatomical 
and  physiological  data,  both  network  models  can  be  aligned  with  brain  stem  and  cerebellar 
circuits  involved  in  classical  NMR  conditioning.  These  models  and  their  implementation  in 
the  brain  have  testable  empirical  consequences. 
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1.  Introduction 


This  chapter  illustrates  how  computational  models  based  on  abstract  neural  networks  can 
provide  insights  into  questions  such  as  where  in  the  brain  learning  occurs  and  the  mechanisms 
that  bring  it  about.  It  serves  as  a  tutorial  on  aligning  quantitative  learning  theory  with 
physiology,  thereby  establishing  a  potential  conduit  for  communication  between  molar  and 
molecular  levels  of  analyses.  One  might  say  that  such  efforts  are  about  bringing  abstract 
models  to  life  in  real  nervous  systems.  Specifically,  it  brings  together  two  lines  of  research: 
(a)  studies  of  brain  circuits  underlying  classical  conditioning  of  a  simple  skeletal  response, 
the  rabbit  nictitating  membrane  response  (NMR)  and  (b)  developing  computational  models 
applicable  to  real-time  conditioning  phenomena  such  as  response  topography  and  CS-US 
interval  effects.  Our  work  stands  on  a  foundation  provided  by  the  “extended  laboratory” 
(Gabriel,  1988)  of  those  who  investigate  classical  eye  blink/NMR  conditioning  (Gormezano, 
Prokasy,  and  Thompson,  1987). 

This  chapter  also  illustrates  Churchland’s  (1986)  point  that  a  theoretical  dimension  can 
be  added  to  the  neurosciences  through  a  process  of  coevolution.  Theories,  expressed  as  com¬ 
putational  algorithms,  can  guide  decisions  about  where  best  to  invest  experimental  resources. 
Behavioral  neuroscientists  regard  the  preparations  they  investigate  as  model  systems,  and 
therefore  they  are  not  indifferent  to  the  broader  implications  of  their  work.  However,  they 
believe  that  exploiting  these  implications  requires  understanding  the  model  system  at  multi¬ 
ple  levels.  At  the  computational  level,  this  understanding  is  expressed  in  terms  of  algorithms 
capable  of  simulating  how  the  system  behaves  under  a  range  of  environmental  challenges. 
Production  systems  have  this  capability.  More  interesting  and  useful  in  the  long  run  are 
algorthims  that  emulate  what  actually  occurs  in  the  brain  to  cause  the  system  behave  as  it 
does. 

I  shall  recount  some  of  the  steps  my  colleagues  and  I  have  taken  in  evaluating  models  of 
classical  conditioning  and  casting  them  in  terms  of  neural  networks  that  might  convincingly 
be  implemented  in  real  brains.  By  this  I  refer  to  discovering  alignments  between  models  and 
what  is  known  of  the  anatomy  and  physiology  underlying  a  given  CR.  I  say  “a  given  CR” 
because  we  are  concerned  primarily  with  developing  theoretical  frameworks  useful  to  neuro¬ 
scientist.  As  neuroscientists,  our  interest  in  theory  cannot  stray  very  far  from  the  nuts  and 
bolts  of  our  experimental  work.  As  neuroscientists,  workers  in  the  extended  NMR  condition¬ 
ing  laboratory  seek  a  deep  understanding  of  classical  conditioning.  Such  an  understanding 
demands  rigorous  theoretical  expression,  and  thus  NMR  conditioning  has  been  approached 
from  three  directions — behavior,  biology,  and  computation.  The  overriding  objective  is  to 
integrate  these  approaches  in  much  the  same  way  that  oculomotor  physiologists  have  done 
with  saccadic  eye  movement  and  vestibular  oculomotor  reflexes  (Robinson,  1981;  1989).  The 
oculomotor  example  is  apt  because  of  shared  anatomical  systems. 


1 


Before  proceeding,  some  comment  about  the  title  is  in  order.  The  title  speaks  of  imple¬ 
menting  connectionist  algorithms1  in  the  brain.  Others  have  applied  the  term  instantiation 
to  this  process.  When  considering  the  dictionary  definitions  of  these  two  terms,  I  find  im¬ 
plementation  more  apt.  According  to  Webster's  New  Collegiate  Dictionary ,  the  infinitive 
“to  instantiate”  means  “to  represent  an  abstraction  by  a  concrete  example”  whereas  “to 
implement”  means  “to  provide  instruments  or  means  of  expression”. 

I  would  argue  that  instantiation  is  a  prerequisite  to  implementation.  To  illustrate,  my 
colleagues  and  I  began  by  selecting  one  abstract  computational  theory,  the  Sutton-Barto 
(SB)  model,  which  showed  promise  in  describing  many  of  the  basic  phenomena  of  classical 
conditioning,  particularly  the  molar  features  of  classical  conditioning  of  the  eye  blink/NMR 
(Barto  and  Sutton,  1982;  Sutton  and  Barto,  1981).  We  then  sought  a  specific  instance  of  the 
SB  model  which  could  describe  in  detail  real-time  features  of  the  conditioned  NMR,  including 
response  topography.  We  have  referred  to  the  resulting  constrained  and  parameterized  ver¬ 
sion  of  the  model  as  the  Sutton-Barto-Desmond  (SBD)  model  (Moore,  Desmond,  Berthier, 
Blazis,  Sutton,  and  Barto,  1986).  By  common  usage,  the  SBD  model  is  an  instantiation — a 
concrete  representation — of  the  abstract  SB  model.  When  we  speak  of  implementing  this 
model  (Moore  and  Blazis,  1989),  we  mean  finding  brain  processes  and  mechanisms  (“instru¬ 
ments”)  conceivably  capable  of  generating  and  explaining  the  measurable  consequences  of 
its  parameterized  and  algorithmic  (“instantiated”)  form. 

Animal  learning  theorists  are  accustomed  to  evaluating  models  solely  by  behavioral  crite¬ 
ria.  They  also  prefer  models  that  are  parsimonious  and  elegant.  Difficulties  arise  when  there 
are  many  serviceable  models  to  account  for  phenomena  at  the  level  of  behavior.  The  instan¬ 
tiation  of  models  into  algorithmic  and  parameterized  form  adds  constraints  to  models  that 
are  useful  in  their  evaluation,  but  even  at  this  level  of  exactness  the  choice  of  the  more  valid 
model  can  be  arbitrary.  Animal  learning  theorists  attempt  to  escape  from  this  dilemma  by 
designing  elaborate  experiments  that  test  highly  refined  behavioral  predictions.  While  not 
minimizing  the  importance  of  this  strategy,  I  suggest  that  a  brain  implementation  scheme 
can  imply  independent  experimental  assessments  of  models  using  criteria  at  the  level  of  neu¬ 
robiology  as  an  adjunct  to  behavioral  criteria.  It  goes  without  saying  that  the  success  of  this 
approach  depends  on  having  accurate  information  about  neural  substrates  of  the  behavior 
being  modeled.  Implementations  schemes  should  be  physiologically  compelling,  not  merely 
plausible.  Later  on  I  will  illustrate  this  point  by  showing  how  implementation  schemes  my 
colleagues  and  I  have  devised  for  two  models  have  testable  hypotheses  in  the  domain  of  neu¬ 
rophysiology.  The  situation  is  somewhat  comparable  to  neuroanatomists’  quest  for  Renhaw 
cells,  which  was  stimulated  by  physiologists’  claims  that  they  must  exist. 

‘Connectionist  algorithms  refer  to  any  member  of  a  class  of  models  cast  in  terms  of  modifiable  connection 
weights  among  neuron-like  processing  elements.  The  term  is  not  limited  to  the  so  called  delta  or  LMS 
learning  rule  or  to  learning  through  back-propagated  error  correction. 
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Devising  good  implementation  schemes  for  instances  of  mammalian  behavior  such  as 
NMR  conditioning  is  not  easy.  It  is  difficult  because  of  the  virtual  impossibility  of  obtaining 
direct  and  rigorous  proof  that  mechanisms  proposed  to  account  for  phenomenology  actually 
exist  and  function  in  ways  consistent  with  the  model.  We  must  therefore  approach  the  ques¬ 
tion  of  mechanisms  indirectly  through  experimental  probes — testing  behavioral  predictions, 
making  lesions  or  pharmacologic  interventions,  and  recording  neural  activity.  It  would  be 
comforting  if  agreed-upon  facts  were  not  open  to  differing  interpretation,  but  this  state  of 
affairs  seldom  exists.  However,  it  is  more  important  to  identify  agreed-upon  experimental 
evidence  than  to  seek  consensus  on  what  the  evidence  means.  What,  then,  of  experimental 
facts  that  are  in  dispute,  and  which  disputes  are  most  crucial  to  resolve?  A  good  model  and 
implementation  scheme  should  tell  us. 


2.  Neurobiological  Background 


Much  of  the  difficulty  in  addressing  questions  of  loci  and  mechanisms  of  learning  arises 
from  a  reliance  on  lesion  data  and  other  indirect  evidence.  Although  lesion  data  are  vitally 
important,  they  have  left  workers  in  this  field  with  some  puzzles  that  will  not  be  easily 
resolved.  These  unresolved  issues  are  important  for  implementing  neural  network  models  of 
NMR  conditioning  in  the  brain.  Before  considering  them,  let  us  review  the  facts  which  I 
believe  are  not  in  dispute. 


•  Telencephalic  brain  regions  and  structures  are  not  essential  for  acquisition,  mainte¬ 
nance,  or  performance  of  the  CR.  Telencephalic  regions  that  have  been  the  focus  of 
lesion  studies  include  neocortex,  hippocampal  formation  and  other  components  of  Pa- 
pez  circuit,  basal  ganglia,  thalamus,  and  hypothalamus.  At  the  midbrain  level,  lesion 
studies  have  shown  that  tectum,  tegmental  reticular  formation,  substantia  nigra,  and 
periacquiductal  grey,  to  mention  a  few  involved  structures,  are  not  essential  for  condi¬ 
tioning.  The  only  essential  midbrain  structure  identified  to  date  is  red  nucleus  (RN). 

•  Metencephalic  brain  regions,  the  cerebellum  and  brain  stem  are  essential  for  the  acqui¬ 
sition,  maintenance,  and  performance  of  CRs.  Within  the  metencephalon,  researchers 
agree  that  the  following  subset  of  structures  are  essential  for  expression  of  robust  CRs 
with  normal  topography:  (a)  Cerebellar  cortex,  specifically  the  region  designated  by 
anatomists  as  hemispheral  lobulus  VI  (HVI);  (b)  cerebellar  nucleus  interpositus,  specif¬ 
ically  the  anterior  region  (NIA);  (c)  magnocellular  RN,  specifically  the  portion  that 
represents  the  facial  region  around  the  eye;  (d)  inferior  olivary  nucleus;  specifically  the 
dorsal  accessory  olive  (DAO)  which  represents  the  facial  regions  around  the  eye  and 
sends  climbing  fibers  to  HVI  and  NIA. 
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•  Lesions  that  disrupt  or  eliminate  CRs  in  one  eye  need  not  affect  acquisition  or  perfor¬ 
mance  of  CRs  by  the  other  eye.  Nor  do  such  lesions  interfere  with  the  normal  savings 
of  trials  to  criterion  when  the  US  is  no  longer  applied  to  the  affected  eye  but  is  switched 
to  the  other  eye.  Since  savings  can  be  demonstrated,  it  is  clearly  the  case  that  les-ons 
do  not  impair  some  general  capacity  to  acquire  and  store  information. 

•  To  a  surprizing  degree,  lesions  that  disrupt  or  eliminate  CRs  do  not  affect  the  UR, 
although  there  may  be  small  modulations  of  UR  amplitude  which  are  understandable 
in  light  of  known  anatomy  and  physiology. 

•  The  reflex  pathways  mediating  the  CR  and  UR  involve  motoneurons  that  innervate  the 
extraocular  muscles,  especially  the  retractor  bulbi  muscles.  Most  retractor  bulbi  mo¬ 
toneurons  lie  in  the  accessory  abducens  nucleus  (AAN)  and  are  innervated  by  nearby 
second-order  sensory  neurons  of  spinal  trigeminal  subnucleus  pars  oralis  (SpoV).  Al¬ 
though  the  brain  stem  components  of  these  circuits  have  been  well  characterized  both 
anatomically  and  physiologically,  the  nature  and  extent  of  the  cerebellum’s  contribu¬ 
tion  remains  an  active  area  of  research.  There  is  strong  evidence,  outlined  below,  that 
the  cerebellum  makes  a  substantial  causal  contribution  to  the  generation  of  CRs. 


Workers  in  the  extended  laboratory  of  the  conditioned  NMR  have  proposed  various  test 
to  determine  whether  CR-disrupting  lesions  or  pharmacologic  interventions  involving  critical 
metencephalic  structures  affect  learning  or  performance.  CR-disruption  might  be  due  to  any 
number  of  factors  unrelated  to  learning — these  might  all  be  subsumed  under  the  heading 
“performance  factors”:  (a)  motor  deficit,  (b)  sensory  deficit,  ruption  of  timing,  (d) 

attentional  deficit.  Motor  deficits  can  be  eliminated  as  a  cause  oi  CR  disruption  to  the 
extent  that  the  UR  remains  unaffected  (over  a  range  of  US  intensities).  Sensory  deficits  can 
be  elimated  to  the  extent  that  CRs  are  disrupted  with  different  CS  modalities.  Attentional 
deficits  can  be  ruled  out  the  extent  that  CRs  occur  in  the  contralateral  eye.  Disruptions  of 
timing  can  be  assessed  by  varying  the  CS-US  interval  (Desmond  and  Moore,  1982). 

Researchers  do  not  agree  on  whether  any  of  the  above  brain  regions  is  involved  in  the 
actual  learning  process  or,  indeed,  whether  learning  also  depends  on  the  integrity  of  other 
structures  (Desmond  and  Moore,  1986).  One  possibility  is  that  “learning  the  connection 
between  a  CS  and  the  US”  occurs  within  Purkinje  cells  (PCs)  of  cerebellar  cortex,  specifically 
at  parallel  fiber  synapses.  Contending  views  are  that  the  critical  connections  are  formed 
within  the  deep  cerebellar  nuclei  or  brain  stem.  It  is  well  to  get  these  issues  in  front  of  us 
before  introducing  implementation  schemes.  Thus  armed,  the  reader  can  better  judge  their 
strengths  and  weaknesses.  Some  unresolved  issues  regarding  the  lesion  data  are  reviewed 
here. 
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•  Do  the  neural  commands  (motor  programs)  that  result  in  a  CR  originate  in  the  cerebel¬ 
lum?  That  is,  can  it  be  said  that  the  cerebellum  is  a  proximal  cause  of  CR  topography? 
We  have  recently  peformed  a  linear  systems  analysis  of  the  relationship  between  the 
firing  rate  of  single  neurons  in  NIA  and  the  position  of  the  NM  during  conditioning 
trials  (Berthier,  Barto,  and  Moore,  1988).  Regarding  a  causative  link,  we  find  that  in 
some  NIA  cells  the  relationship  rate  of  firing  and  position  of  the  NM  can  be  modeled  by 
a  nonrecursive  (causal)  digital  filter  (Hamming,  1983).  Using  a  related  approach,  we 
estimate  the  transfer  function  between  neural  activity  and  NM  position,  and  from  this 
it  is  possible  to  write  a  differential  equation  relating  the  two  variables.  Some  NIA  cells 
with  CR-predictive  firing  yield  equations  for  second-order  systems.  These  equations 
relate  a  cell’s  firing  rate  to  the  acceleration,  position,  and  velocity  of  NM  movement. 
It  is  worth  noting  that  NM  movement  is  linearly  related  to  eyeball  retraction.  In  fact, 
the  sweep  of  the  NM  over  the  eye  is  caused  by  retraction  of  the  globe.  The  mechanics 
of  eyeball  retraction  are  those  of  a  Voight  element  consisting  of  elastic  and  viscous 
components,  a  classic  textbook  example  of  a  second-order  linear  system.  Hence,  it 
is  perhaps  not  surprizing  that  equations  relating  the  firing  of  some  NIA  cells  to  the 
conditioned  NMR  are  those  of  second-order  linear  systems. 

•  Do  lesions  of  cerebellar  cortex  produce  a  complete  and  permanent  loss  of  a  previously 
acquired  CR?  This  question  is  important  in  itself,  but  especially  so  because  of  the  well 
known  theories  of  Marr  (1969)  and  Albus  (1971),  who  independently  suggested  that 
motor  learning,  including  classical  conditioning,  involves  cerebellar  cortex  in  a  funda¬ 
mental  way.  Yeo  and  his  colleagues  were  the  first  to  show  that  HVI  lesions  disrupt 
CRs,  typically  by  either  eliminating  them  altogether  or  greatly  reducing  their  ampli¬ 
tude  (Yeo,  Hardiman,  and  Glickstein,  1984).  These  studies  have  been  extended  to 
show  that  lesions  of  HVI  must  be  complete  in  order  to  eliminate  small  amplitude  re¬ 
sponses  that  would  normally  be  counted  as  CRs  and  to  prevent  recovery  after  extended 
post-operative  training  (Yeo  and  Hardiman,  1988).  Lavond,  Steinmetz,  Yokaitis,  and 
Thompson  (1987)  report  small  amplitude  CRs  and  recovery  following  extended  training 
in  cases  where  HVI  removal  appears  to  have  been  complete,  and  so  the  issue  remains 
in  question. 

•  Do  lesions  of  the  inferior  olivary  nuclei  cause  a  progressive  decline  of  the  CR  resembling 
extinction  as  some  investigators  claim  (McCormick,  Steinmetz,  and  Thompson,  1985), 
or  is  it  the  case  that  such  lesions  merely  disrupt  normal  functioning  cerebellar  in  this 
way  bring  about  an  immediate  detrimental  effect  on  performance  (Yeo,  Hardiman,  and 
Glickstein,  1986)?  This  issue  is  as  yet  unresolved  because  experiments  have  yielded 
conflicting  results.  If  lesions  of  the  inferior  olivary  nuclei,  particularly  DAO,  do  result 
in  experimental  extinction,  then  this  would  support  models  built  on  the  idea  that 
climbing  fibers  from  DAO  to  cerebellum  carry  the  reinforcement  signal  from  the  US  to 
sites  of  learning. 
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•  Does  stimulation  of  the  DAO  provide  a  reinforcing  signal  for  learning  in  cerbellar  cortex 
(Steinmetz,  Lavond,  and  Thompson,  1989)?  Or  is  any  learning  in  the  cerebellum 
confined  to  NIA  which  receives  climbing  fiber  collaterals  from  DAO?  Related  to  this 
question  is  the  possibility  that  stimulation  of  DAO  does  not  reinforce  learning  in  the 
cerebellum  at  all.  It  may  merely  stimulate  brain  stem  elements  of  the  reflex  pathway 
underlying  the  UR  where  learning  might  occur  (Bloedel,  1987;  Yeo,  1989). 


There  are  other  issues  to  be  resolved  among  workers  in  the  extended  laboratory  of  the 
conditioned  NMR.  I  have  mentioned  only  those  that  bear  on  evaluating  the  implementation 
schemes  suggested  for  each  of  the  models  outlined  in  subsequent  sections.  Each  model  has 
its  own  implementation  scheme.  They  both  assume  that  learning  occurs  through  Hebbian 
mechanisms  which  involve  synaptic  modification  through  convergence  of  CS  and  US  informa¬ 
tion  onto  single  neurons  (Byrne,  1987).  They  both  assume  that  these  synaptic  modifications 
occur  in  cerebellar  cortex  (HVI)  and  that  CRs  are  initiated  by  the  action  of  PCs  on  NIA  cells 
to  which  they  project.  The  implementation  for  the  SBD  model  assumes  that  learning  oc¬ 
curs  in  cerebellar  cortex,  but  one  synapse  before  the  PC  or  output  stage  (Moore  and  Blazis, 
1989).  The  other  model,  designated  VET,  is  more  complex  and  assumes  that  learning  occurs 
within  the  brain  stem  as  well  as  in  cerebellar  cortex.  The  cortical  component  assumes  that 
learning  occurs  through  modification  of  parallel  fiber/PC  synapses.  In  addition,  it  assumes 
that  learning  also  occurs  at  the  synapses  of  parallel  fibers  and  Golgi  cells  (Moore,  Desmond, 
and  Berthier,  1989). 


3.  The  SBD  Model 


The  SBD  model  has  been  described  in  detail  elsewhere  (Moore,  Desmond,  Berthier, 
Blazis,  Sutton,  and  Barto,  1986;  Moore  and  Blazis,  1989)  and  so  a  brief  summary  will 
suffice.  The  model  is  based  on  a  single  neuron-like  processing  unit  which  receives  input  from 
many  potential  CSs  and  a  US.  The  processing  unit  adjusts  the  weights  (synaptic  efficacies) 
of  the  CS  input  so  that  future  output  of  the  unit  matches  its  current  output. 

Equation  1  specifies  that  the  output  of  this  system,  denoted  s(£),  equals  the  weighted 
sum  of  its  input  from  potential  CSs  and  the  US.  The  variable  s(£)  is  a  linear  function  of 
its  weighted  input  only  within  an  allowed  range  imposed  by  the  fact  that  the  NM  can  only 
move  so  far  (about  10  millimeters)  as  the  eyeball  retracts.  A  CS’s  contribution  to  the  output 
of  the  element  is  the  product  the  current  strength  of  its  representation,  denoted  x(t)  in  the 
model,  and  a  corresponding  “synaptic"  weight  denoted  V(t).  Formally,  the  output  of  the 
system  at  time  t,  denoted  a(£),  equals  the  weighted  sum  of  input  from  all  CSs,  where  x<(£ ) 
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refers  to  the  magnitude  of  CS;,  i  =  1,  ...,n,  at  time  t: 


*(o  =  e  kmm  t) + ao- 

1=1 


(1) 


A'(t)  is  the  US’s  contribution  to  s(£). 

In  Equation  1,  x;(£)  represents  the  activation  level  of  the  ith  member  of  a  set  of  potential 
CSs  at  discrete  times  t  after  onset  ( t  represents  successive  time  steps  of  10- milliseconds  du¬ 
ration).  The  following  specifications  for  x;  were  dictated  by  two  contraints:  (a)  generation  of 
realistic  response  topography  for  a  forward-delay  paradigm  with  a  favorable  CS-US  interval, 
and  (b)  generation  of  realistic  interstimulus  interval  (ISI)  functions.  The  optimal  CS-US 
interval  for  NMR  conditioning  is  generally  taken  to  be  250  milliseconds  (Gormezano,  1972). 
When  the  CS;  begins,  xt-  =  0.0.  Jt  remains  at  0.0  until  t  =  7,  i.e.,  70  milliseconds  after  CS; 
onset.  At  this  point,  X;  begins  to  increase  in  an  S-shaped  fashion.  It  levels  off  to  a  maximum 
value  of  1.0  by  t  =  30  (300  milliseconds  after  CS,'  onset)  and  remains  at  this  value  until  CS; 
offset,  at  which  time  i;  begins  to  fall  exponentially  back  to  0.0.  Thus,  according  to  Equation 
1  the  output  of  the  model,  s(t),  to  CS;  conforms  to  the  temporal  map  or  template  provided 
by  x;.  As  the  number  of  training  trials  increases,  the  variable  V;(t)  increases,  and  the  CR 
becomes  increasingly  robust.  This  process  is  reversed  over  a  series  of  extinction  trials. 

Learning  in  the  SBD  model  follows  a  modified  Hebbian  rule  which  states  that  changes 
of  the  synaptic  weight  of  CS;,  AV;,  are  proportional  to  the  difference  between  the  current 
output,  s(t),  and  the  trace  of  preceding  outputs,  S(t).  At  time  t,  AV;  is  computed  as  follows: 

A  Vi(t)  =  c[s(t)  -  $(t)]x;(*),  (2) 

where  c  is  a  learning  rate  parameter,  0  <  c  <  1. 

The  factor  x;(£)  in  Equation  2  specifies  the  degree  to  which  the  “synaptic  junction” 
corresponding  to  CS;  is  eligible  for  modification  (Sutton  and  Barto,  1981).  x;  is  driven  by 
the  variable  x(i);:  After  CS;  onset,  it  increases  with  the  x;,  but  with  a  lag  of  30  milliseconds. 
It  remains  at  full  strength  as  long  as  the  CS;  is  on  and  begins  to  decay  to  a  baseline  value 
of  zero  30  milliseconds  after  CS;  offset.  The  rate  of  this  decay  is  inversely  related  to  CS; 
duration  whenever  CS;  exceeds  250  milliseconds. 

Equation  2  does  not  contain  an  explicit  term  for  the  reinforcing  action  of  the  US.  The 
US  is  important  only  insofar  as  it  affects  the  term  s(£)  —  s(t).  The  interaction  the  two  time- 
dependent  variables  associated  with  CS; ,  x;(£)  and  X;,  together  with  s(t)  —  s(t ),  govern  the 
rate  of  learning  and  shape  of  ISI  functions.  Trial-wise  learning  curves  reflect  accummulated 
net  changes  in  V;  occurring  within  each  trial.  Such  changes  occur  before  and  after  the 
occurrence  of  the  US.  For  example,  given  the  10- millisecond  time  step  used  in  our  simulations 
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(e.g.,  Moore  et  al,  1986),  a  trial  with  an  ISI  of  350  milliseconds  might  involve  over  400 
computations  of  AV*. 

The  term  s(f)  in  Equation  2  can  be  thought  of  as  a  short-term  decaying  trace  of  the 
system’s  output  from  previous  time  steps.  Alternatively,  it  can  be  interpreted  as  a  prediction 
or  expectation  of  output  based  on  previous  output.  It  is  computed  as  follows: 

s(t  +  1)  =  (3a{t)  +  (1  -  /3)s(t),  (3) 

where  0  <  (3  <  1. 

Simulation  studies  indicate  that,  with  a  10-milliseconds  time  step,  optimal  performance 
of  the  model  requires  that  f3  be  on  the  order  of  0.6  (Blazis  and  Moore,  1987).  Equation  3 
with  =  0.6  also  describes  the  inhibitory  action  of  cerebellar  Golgi  cells  on  information 
flow  through  the  granular  layer  of  cerebellar  cortex.  This  coincidence  was  exploited  in 
implementing  the  model  (Moore  and  Blazis,  1989). 

The  foregoing  assumptions  enable  the  SBD  model  to  simultaneously  generate  response 
topographies  and  ISI  functions  for  both  trace  and  forward-delay  conditioning  paradigms 
(Moore  et  al,  1986).  In  addition,  it  retains  the  original  Sutton-Barto  model’s  ability  to  de¬ 
scribe  multiple-CS  phenomena  such  as  blocking,  higher-order  conditioning,  and  conditioned 
inhibition  (Barto  and  Sutton,  1982;  Sutton  and  Barto,  1981).  In  agreement  with  experimen¬ 
tal  liturature  (Miller  and  Spear,  1985),  the  model  does  not  predict  extinction  of  conditioned 
inhibition. 


3.1  Implementing  the  SBD  model 

Moore  and  Blazis’s  (1989)  implementation  of  the  SBD  model  is  summarized  in  this  sec¬ 
tion.  We  required  that  the  implementation  scheme  meet  the  following  criteria:  (a)  It  had 
to  involve  the  cerebellum  and  be  consistent  with  its  anatomy  and  physiology,  (b)  It  had 
to  account  for  the  lesion  data,  namely  the  fact  that  lesions  of  cerebellar  cortex  (HVI)  or 
associated  brain  stem  circuits  virtually  elimiate  the  CR.  (c)  It  had  to  propose  neuronal  loci 
where  the  learning  rule  (Equation  2)  might  be  implemented.  This  meant  proposing  sites 
where  CS  information,  the  variable  x,  convergences  with  the  reinforcement  signal,  3  —  5.  (d) 
It  had  to  propose  schemes  for  computing  s  via  Equation  3  and  s  —  5.  The  last  item  held  the 
key.  Once  overcome,  the  rest  of  the  implementation  fell  into  place. 

Turning  now  to  the  details  of  the  implementation,  the  output  variable  3,  which  expresses 
the  form  of  the  CR  and  is  used  in  the  learning  rule,  is  generated  by  the  action  HVI  PCs  on 
neurons  in  NIA  to  which  they  project.  Evidence  supporting  this  construction  was  reviewed 
above.  Next,  3  is  transmitted  with  high  fidelity  through  each  of  the  synaptic  links  leading  to 
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AAN  motoneurons  and  generation  of  the  peripherally  observed  CR.  One  synapse  before  this 
stage,  within  SpoV,  an  efference  copy  of  3  peels  off  and  ascends  back  to  HVI  via  mossy  fibers. 
This  efference  copy  of  3  consists  of  two  streams.  One  stream  passes  through  the  granular 
layer  without  modulation  by  Golgi  cells.  This  stream  gives  rise  to  parallel  fibers  (axons  of 
granule  cells)  that  carry  3  information  to  other  circuit  components,  including  Golgi  cells  that 
modulate  the  other  s  stream.  This  modulation  computes  3  and  gives  rise  to  parallel  fibers 
carrying  5  information. 

The  existence  of  separate  parallel  fibers  which  carry  3  and  5  information  allow  for  com¬ 
putation  of  the  reinforcement  factor  in  Equation  2.  Golgi  cells,  different  from  those  that 
compute  s,  receive  two  simultaneous  inputs:  an  excitatory  input  from  5-carrying  parallel 
fibers  and  an  inhibitory  input  from  axon  collaterals  of  PCs  activated  by  3-carrying  parallel 
fibers.  Because  Golgi  cells  are  inhibitory  neurons,  the  resulting  action  on  granule  cells  to 
which  they  project  is  proportional  to  3  —  5.  In  the  implementation  scheme,  this  output  is 
directed  to  granule  cells  activated  by  CSs  via  mossy  fibers.  CS  information  has  been  pre- 
processed  such  that  it  expresses  the  variable,  x.  The  eligibility  factor,  x,  presumably  resides 
within  x-activated  granule  cells.  They  are  sites  of  convergence  for  implementing  of  the  learn¬ 
ing  rule  because  they  encode  x  and  receive  synaptic  input  encoding  the  reinforcement  factor, 
3  —  5,  from  Golgi  cells. 

Having  computed  AV,  the  output  of  these  granule  cells  on  the  next  computational  step 
is  proportional  to  Vx,  which  is  also  equal  to  the  next  value  of  s  if  we  assume  only  one  CS. 
Parallel  fibers  carrying  Vx  impinge  on  basket  cells  and  PCs.  The  basket  cells,  which  are 
inhibitory  neurons,  send  axons  to  PCs  on  an  adjacent  group  of  parallel  fibers.  Increasing 
activation  of  basket  cells  causes  the  firing  of  these  PCs  to  decrease  below  their  baserate  of 
firing  in  proportion  to  Vx.  This  decrease  in  firing  disinhibits  neurons  in  NIA  to  which  they 
project,  causing  them  to  increase  their  firing  rate  and  send  an  excitory  pulse  of  activation 
through  the  efferent  pathway  which  terminates  in  AAN. 


3.2  Implication  of  the  SBD  model 

The  complexity  of  the  implementation  scheme  stems  from  considerations  of  anatomy  and 
physiology.  These  considerations  are  elaborated  elsewhere  (Blazis  and  Moore,  1987;  Moore 
and  Blazis,  1989).  The  most  compelling  evidence  for  the  scheme  comes  from  Berthier  and 
Moore’s  (1986)  study  of  how  PCs  in  HVI  respond  during  NMR  conditioning. 

Berthier  and  Moore  (1986)  recorded  from  single  PCs  in  HVI  during  the  asymptotic  stages 
of  two-tone  differential  conditioning.  PCs  with  CR-related  firing  patterns  could  be  classified 
as  either  increasing  or  decreasing  their  baserate  of  firing  whenever  a  CR  occurred.  Three 
cells  increased  firing  for  every  one  that  decreased  firing.  Firing  patterns  of  PCs  could  also 
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be  classified  as  either  preceding  or  occurring  simultaneously  with  CRs.  All  of  the  PCs  that 
decreased  their  firing  did  so  before  the  CR  occurred,  as  would  be  necessary  if  their  activity 
were  responsible  for  initiating  CRs.  PCs  which  increased  their  firing  before  the  occurrence 
of  CRs  are  implied  by  parallel  fibers  carrying  Vx  information.  PCs  that  increase  their  firing 
simultaneously  with  CRs  are  implied  by  the  two  streams  of  s-carrying  efference  from  SpoV 
used  in  the  computation  of  5  —  a.  It  remains  to  be  determined  whether  we  can  objectively 
discriminate  increases  in  PC  firing  that  mirrors  3  from  that  mirroring  s.  This  caveat  aside, 
the  frequency  distribution  of  firing  patterns  observed  by  Berthier  and  Moore  (1986)  are 
accounted  for  by  the  implementation  scheme. 


3.2.1  Behavioral  predictions 

One  prediction  of  the  SBD  model  is  that  the  rate  of  CR  acquisition  in  a  trace  conditioning 
paradigm  (but  not  a  forward-delay  paradigm)  is  an  increasing  function  CS  duration,  provided 
the  interval  between  CS  offset  and  US  onset  (trace  interval)  is  sufficiently  long,  e.g.,  300 
milliseconds  or  more.  The  prediction  follows  directly  from  the  model’s  assumption  that  the 
eligibility  factor  in  the  learning  rule,  x,  decays  at  a  rate  which  is  inversely  related  to  the 
duration  of  the  CS.  The  prediction  is  counterintuitive  because  it  states  that  acquisition  can 
be  faster  with  a  longer-than-optimal  ISI  than  one  nearer  to  optimal.  Preliminary  studies 
with  an  acoustic  CS  (Blazis  and  Moore,  1989)  have  born  out  this  prediction,  but  only  when 
the  CS  is  sufficiently  intense  (e.g.,  80  dB).  The  prediction  is  not  supported  with  a  CS  on  the 
order  of  60  dB.  In  this  case  acquisition  rate  appears  to  be  dominated  not  by  CS  duration 
but  as  in  forward-delay  conditioning  by  ISI. 

I  mention  this  particular  experiment  in  order  point  out  that  the  SBD  model  does  not 
yet  provide  a  complete  account  of  NMR  conditioning.  In  this  case,  the  model  is  incomplete 
because  it  says  nothing  about  how  CS  intensity  affects  the  variables  x  or  x.  Part  of  the 
motivation  for  the  above  experiment  was  to  obtain  information  on  how  to  incorporate  these 
effects  into  the  model.  The  SBD  model  also  fails  to  take  account  of  processes  that  might 
occur  within  intertrial  intervals,  e.g.,  the  “consolidation”  effects  from  studies  showing  that 
rate  of  conditioning  is  a  direct  function  of  intertrial  interval  (Moore  and  Gormezano,  1977). 


3.2.2  Physiological  predictions 

The  most  important  prediction  from  the  implementation  scheme  is  that  learning  occurs 
within  the  granular  layer  of  cerebellar  cortex.  Testing  this  prediction  will  require  experiments 
on  cerebellar  slices  using  designs  similar  to  those  employed  by  Coulter  and  Disterhoft  to 
investigate  long-term  effects  of  NMR  conditioning  on  hippocampal  pyramidal  cells  (e.g., 
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Coulter,  Lo  Turco,  Kubota,  Disterhoft,  Moore,  and  Alkon,  1989).  It  also  remains  to  be 
proven  that  SpoV  cells  that  fire  in  relation  to  CRs  actually  send  collateral  efference  copy 
mirroring  CR  waveform  to  HVI.  All  we  can  say  with  confidence  at  this  point  is  that  cells  in 
SpoV  exist  that  show  CR-related  firing  of  the  kind  needed  to  provide  HVI  with  a  current 
copy  of  the  variable  s  (Ricciardi,  Richards,  and  Moore,  1989).  We  also  know  that  some  SpoV 
cells  send  mossy  fibers  to  HVI.  We  no  not  know  that  the  two  categories  actually  overlap. 

The  implementation  scheme  does  not  assign  a  role  in  learning  to  climbing  fiber  inputs 
from  DAO.  Additional  studies  are  needed  to  determine  whether  this  is  justified.  One  reason 
for  discounting  the  possible  contribution  of  climbings  fibers  is  that  their  low  firing  rate 
makes  them  poor  candidates  for  providing  efference  copy  about  s  for  implementation  of  the 
learning  rule.  In  addition,  very  few  PCs  in  the  Berthier  and  Moore  (1986)  responded  with 
a  complex  spike,  indicative  of  climbing  fiber  input,  when  the  US  occurred.  Nevertheless, 
the  possibility  that  climbing  fibers  are  important  for  learning  cannot  be  ruled  out  without 
further  experimental  work  (Moore  and  Berthier,  1987;  Ito,  1989).  In  fact,  Moore  et  al’s  (1989) 
implementation  of  the  VET  model,  reviewed  in  the  next  secton,  assumes  that  learning  in 
the  cerebellum  is  reinforced  by  climbing  fiber  input  elicited  by  the  US. 


4.  The  VET  Model 


The  SBD  model  is  able  to  simulate  response  topography  because  it  assumes  that  every 
potential  CS  provides  the  system  with  a  fixed  pattern  of  activation  which  serves  as  a  template 
for  the  CR.  We  have  speculated  about  how  template  may  be  formed  and  the  the  possible 
contributions  of  other  brain  regions,  especially  the  hippocampus,  in  this  process  (Blazis 
and  Moore,  1987).  The  fact  remains  that  the  SBD  model  basically  sidesteps  this  issue. 
In  addition,  the  SBD  model  is  incapable  of  adaptively  changing  CR  topography  so  as  to 
simulate  a  number  of  paradigms.  The  SBD  model  cannot  yield  appropriate  CR  waveforms 
for  the  following  cases: 


•  Trace  conditioning.  CR  waveforms  should  peak  just  before  the  occurrence  of  the  US, 
as  in  forward-delay  paradigms,  but  the  dynamics  of  the  variable  x  do  not  permit  this 
to  happen.  Instead,  CR  waveforms  begin  to  fall  toward  baseline  when  CS  offset  occurs. 

•  Forward-delay  conditioning  with  long  CS-US  intervals.  The  SBD  model  cannot  sim¬ 
ulate  inhibition  of  delay.  Since  CR-waveform  mirrors  the  template  provided  by  the 
variable  x,  its  latency  and  form  are  not  influenced  by  CS-US  interval. 

•  Multiple  CS-US  intervals.  Training  with  multiple  CS-US  intervals  yields  complex 
CR-waveforms.  For  example,  a  study  by  Millenson,  Kehoe,  and  Gormezano  (1977) 
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showed  that  training  with  randomly  mixed  trials  having  CS-US  intervals  of  200  and 
700  milliseconds  gave  rise  to  CRs  with  two  peaks,  each  centered  at  a  point  of  US  onset. 


The  VET  model  overcomes  these  deficiencies.  It  is  able  to  simulate  appropriate  CR 
waveforms  for  these  cases  because  of  features  absent  in  the  SBD  model.  These  features 
include  the  following: 


•  CSs  are  provided  with  a  temporal  dimension  through  tapped  delay  lines  which  encode, 
not  only  the  source  of  the  stimulus,  but  also  the  time  since  the  stimulus  began.  Another 
set  of  tapped  delay  lines  encodes  the  time  since  the  stimulus  ceased.  Hence,  the  model 
has  timed-tagged  input  elements  for  both  stimulus  onset  and  offset  (Equation  4). 

•  There  are  two  neuron-like  processing  units  that  receive  convergent  input  from  CSs 
and  the  US.  One  unit  (designated  V)  is  the  output  device  (Equation  5).  It  has  mod¬ 
ifiable  synaptic  weights  which  are  changed  according  to  an  LMS  rule  resembling  the 
Resorla- Wagner  model  (Equation  6).  The  main  difference  from  the  Rescorla- Wagner 
model  is  that  weight  changes  depend  on  local  eligibility  factors  (Equation  7),  a  global 
ISI  parameter  (Equation  8),  and  on  an  additional  reinforcement  signed  reflecting  the 
expected  time  of  occurrence  of  the  US  (Equation  9). 

•  This  additional  reinforcement  signal  is  computed  by  the  other  processing  unit,  des¬ 
ignated  the  E  unit,  which  learns  when  the  US  occurs.  Like  the  V  unit,  the  E  unit 
receives  convergent  input  from  CSs  and  the  US,  and  it  has  modifiable  synaptic  weights 
which  are  changed  according  to  a  simple  linear  difference  equation  (Equation  11)  which 
includes  local  eligibility  factors  (Equation  10)  and  the  global  ISI  parameter  defined  in 
Equation  8. 


A  more  formal  treatment  follows: 

Architectural  assumptions  for  the  network  have  been  described  in  detail  elsewhere  (Desmond 
and  Moore,  1988;  Moore,  Desmond,  and  Berthier,  1989).  Basically,  the  onset  and  offset  of 
each  CS  begin  activation  of  separate  tapped  delay  lines.  The  elements  in  the  delay  line  are 
referred  to  as  x^  elements  because  each  element  can  be  referenced  by  (a)  its  CS  (i),  (b) 
whether  it  is  activated  by  the  onset  ( j  =  1)  or  offset  ( j  =  0)  of  the  CS,  and  (c)  its  number 
within  the  delay  line  ( k ).  For  example,  element  X20S  belongs  to  the  offset  delay  line  for  CS2, 
and  is  the  eighth  element  activated  in  the  delay  line.  The  output  of  an  Xjjfc  element  (which 
is  either  1  or  0)  at  time  t  is  designated  x^(t). 

The  Xijk  tapped  delay  line  elements  are  activated  sequentially,  with  a  new  element  re¬ 
cruited  every  time  step  (10  ms).  When  activated,  an  xtJ*  element  changes  value  from  0  to 
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1,  and  remains  at  1  for  10  time  steps.  Thus,  for  each  trial  beginning  at  time  t  =  1: 


Xijk{t)  = 


1  if  Tij  -j-  k  —  1  <  t  <  Tij  +  k  -j-  9; 
0  otherwise, 


(4) 


where  is  the  onset  time  (j  =  1)  or  offset  time  ( j  =  0)  of  CS  i  (x;j*.  =  0  if  CS  t  is  not 
presented). 


In  addition  to  the  x,j*.  elements,  the  network  has  two  higher-order  processors  designated 
the  V  unit  and  E  unit.  Each  element  gives  off  two  taps,  one  of  which  projects  to  the  V 
unit  and  the  other  to  the  E  unit.  The  latter  connections  are  modifiable,  and  the  weights  for 
these  connections  are  referred  to  as  Vijt t  and  Eijk ■  All  other  connections  are  non-modifiable; 
these  include;  US  connections  with  the  V  and  E  units,  an  E-unit  projection  to  the  V  unit, 
and  connections  between  adjacent  Xijk  elements  in  the  delay  line. 

The  output  of  the  network,  s(t),  is  derived  from  the  US  input  and 
sum  of  the  V  unit  inputs,  and  is  defined  as; 

a(0  =  £  £  £  Vijk(t)xijk(t)  +  L(t), 

»  j  k 

where  s(t)  is  confined  to  the  closed  unit  interval. 

Changes  in  the  V{jk  weights  are  given  by  the  following  expression: 

AVriik(t)  =  c{L{t)  -  s(f)}/iijfc(t)xij(t)r(t).  (6) 


from  the  weighted 

(5) 


where: 


c  is  a  rate  parameter,  0  <  c  <  1. 

L{t)  is  the  reinforcement,  0  <  L(t)  <  A,  where  A  is  analogous  to  the  strength  of  the  rein¬ 
forcement  during  conditioning,  0  <  A  <  1.  L(t)  =  0  until  the  US  occurs,  at  which  time 
L{t)  =  A. 

•s(t)  =  £  X)  £  and  is  confined  to  the  closed  unit  interval. 

hi jfc(t)  constitutes  an  eligibility  trace  for  each  Kj*  synapse,  0  <  htJ*(t)  <  1.  This  term  has 
maximum  value  at  the  onset  time  of  the  element  and  decays  geometrically.  It  is  com¬ 
puted  as  follows: 


hijk{t)  =  < 


1.0 

(0.%)hijk(t  -  1) 

0.0 


if  t  =  +  k  -  1; 
if  t  >  +  k  —  1; 
otherwise, 


(7) 
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Xij(t)  is  an  overall  CS  eligibility  for  onset  and  offset  processes,  0  <  xy(t)  <  1.  This  function 
governs  the  rate  of  conditioning  that  occurs  at  a  given  interstimulus  interval.  xy(t)  is 
globally  available  to  all  Ky*  and  Eijk  synapses.  The  equations  described  below  approx¬ 
imate  the  empirically  observed  inverted-U-shaped  function  found  in  rabbit  nictitating 
membrane  conditioning: 

(  (.05)(i  -  77 j)  -  0.25  if  +  6  <  t  <  Tij  +  25; 

*i,-(i)  =  <  ( —  1/ 475)(t  -  m)  +  (500/475)  if  ry  +  25  <  t  <  ry  +  500;  (8) 

(  0.0  otherwise. 

r(f)  is  the  output  of  the  E  unit,  and  represents  the  temporal  expectation  of  reinforcement, 
0  <  r(t)  <  A.  It  is  defined  as  follows: 


w 


here: 


r(t)  =  ma.x{Eijk{t)Axijk{t)  |  i  =  1, . . .  ,n;  j  =  0, 1;  k  =  I, . . . ,  N) 


**■-*«={  o  o,h^L;^(‘"1)=1; 


(9) 


(10) 


and  E^k  are  the  connection  weights  of  the  input  elements  onto  the  E  unit.  Changes  in 
these  weights  are  given  by: 


AEijk{t)  =  c[L{t )  -  r(t)]AiljA(f)iii(t) 


(11) 


4.1  Implementing  the  VET  model 

The  implementation  criteria  for  the  VET  model  were  the  same  as  for  the  SBD  model.  The 
two  schemes  share  many  common  features,  e.g.,  they  both  assume  that  CRs  are  generated 
by  the  disinhibiting  action  of  HVI  PCs  on  NIA  neurons,  and  Golgi  cells  play  a  crucial  role  in 
both.  The  scheme  assumes  that  the  time-tagged  input  elements  associated  with  a  CS  ascend 
to  cerbellar  cortex  via  mossy  fibers.  They  synapse  within  the  granule  layer,  and  with  no 
modulation  are  assigned  to  a  corresponding  set  of  parallel  fibers.  Note  that  the  tapped  delay 
line  architecture  exists  outside  the  cerebellum,  and  their  anatomical  justification  is  discussed 
elsewhere  (Moore  et  al,  1989).  The  parallel  fibers  which  carry  CS  information  synapse  on 
both  PCs  and  Golgi  cells. 

These  synapses  are  modifiable.  Synapses  on  PCs  are  sites  where  Equation  6  is  imple¬ 
mented,  and  hence  these  cells  are  the  V  units.  Synapses  on  Golgi  cells  are  sites  where 
Equation  11  is  implemented,  and  hence  these  cells  are  the  E  units.  Both  sets  of  modifiable 
synapses  are  changed  when  the  US  occurs  to  the  extent  that  they  are  eligible.  The  US  trig¬ 
gers  a  climbing  fiber  volley  which  causes  these  synapses  to  undergo  synaptic  depression  via 
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mechanisms  of  long  term  dression  (LTD)  identified  by  Ito  (1989)  and  others.  With  learning, 
CS  input  causes  V  and  E  units  to  decrease  their  baserate  of  firing.  For  V  units,  this  decrease 
initates  a  CR.  For  E  units,  this  decrease  disinhibits  certain  granule  cells.  These  granule  cells 
receive  input  from  brain  stem  neurons,  possibly  in  SpoV,  which  also  undergo  learning  due  to 
the  convergence  of  CS  and  US  information.  Although  no  learning  rule  is  specified,  the  SBD 
(or  SB)  model  suffices  provided  the  variable  x  in  Equation  1  increases  rapidly  to  a  plateau 
at  CS  onset  and  decays  slowly  at  CS  offset. 

With  learning,  the  output  (rate  of  firing)  of  these  brain  stem  units  consists  of  a  short 
latency  plateau  which  extends  well  beyond  the  offset  of  the  CS.  This  output  provides  a 
uniform  and  long  lasting  stream  of  excitatory  input  to  granule  cells.  The  E  units,  however, 
gate  this  excitation  and  prevent  it  from  exciting  parallel  fibers  that  synapse  on  the  V  units. 
As  learning  builds  up  in  the  E  units,  this  gate  is  opened,  but  only  during  times  near  the 
expected  occurrence  of  the  US.  Thus,  E  units  provide  the  second  reinforcing  event  necessary 
for  modification  of  CS-input  synapses  on  the  V  units  so  that  they  express  appropriately 
timed  CR  waveforms. 


4.2  Implications  of  the  VET  model 

The  VET  model  simulates  virtually  all  of  tb  behavioral  phenomena  encompassed  with 
the  SBD  model,  but  unlike  that  model,  it  has  the  added  feature  of  predicting  realistic  S- 
shaped  acquisition  curves.  Unlike  the  SBD  model,  however,  it  cannot  simulate  second-order 
conditioning.  As  in  the  Rescorla- Wagner  model,  connection  weights  are  strengthened  only 
in  the  presence  of  the  US.  Desmond  and  Moore  (1988)  describe  several  novel  predictions 
of  the  VET  model.  One  prediction  is  that  lengthening  the  duration  of  a  CS  after  trace 
conditioning  should  result  in  double-peaked  CR  waveforms,  reflecting  contributions  of  both 
onset  and  offset  tapped  delay  elements.  The  weights  of  offset  elements  are  normally  masked 
by  those  of  the  onset  elements,  but  lengthening  the  CS  exposes  these  offset  elements,  and 
CRs  with  two  peaks  emerge. 

The  physiological  implications  of  the  implementation  scheme  are  many.  The  most  inter¬ 
esting  implication  is  the  possibility  that  cerebellar  Golgi  cells  express  LTD  at  parallel  fiber 
synapses  in  a  manner  analogous  to  that  of  PCs.  There  appears  to  be  no  evidence  on  this 
point  in  the  literature.  Another  implication  of  the  scheme  is  that  climbing  fibers  from  DAO 
do,  in  fact,  reinforce  learning.  In  order  for  this  possibility  to  be  taken  seriously,  it  would 
be  necessary  to  record  from  PCs  (and  Golgi  cells)  in  HVI  during  the  initial  stages  of  CR 
acquisition.  This  was  not  done  in  the  Berthier  and  Moore  (1986)  study,  nor  in  any  other 
study,  for  methodological  reasons:  It  is  virtually  impossible  to  record  from  single  neurons  in 
an  awake  animal  for  the  hundreds  of  trials  normally  required  to  obtain  robust  CRs.  Never- 
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theless,  there  are  ways  around  this  problem  we  are  pursuing  in  our  laboratory.  Finally,  the 
scheme  implies  the  existence  of  neurons  in  the  brain  stem  that  project  to  HVI  and  fire  in 
accordance  with  the  scheme’s  requirements.  We  have  seen  a  number  of  candidate  neurons 
in  recordings  from  cells  in  NIA,  RN,  SpoV.  Cells  in  RN  and  SpoV  project  to  HVI,  and  it 
is  possible,  though  not  proven  in  rabbit,  that  NIA  cells  send  mossy  fibers  to  HVI  as  axon 
collaterals. 


5.  Concluding  Remarks 


The  two  models  and  implementation  schemes  are  quite  different,  but  not  mutually  exclu¬ 
sive.  The  VET  model  implies  a  neural  network  architecture  which  is  well  suited  for  forming 
appropriate  CR  waveforms  for  a  wide  range  of  circumstances.  The  output  of  this  network 
need  not  be  regarded  as  input  to  motoneurons.  Instead,  it  could  be  regarded  as  a  template 
used  by  another  learning  system  responsible  for  such  things  as  generating  CRs  and  imple¬ 
menting  second-order  conditioning.  Such  a  system  might  resemble  the  SBD  model  or  some 
other  supervised  learning  (error  correction)  algorithm. 

Such  a  hybrid  model  would  not  encompasses  phenomena  which  both  models  fail  to  ad¬ 
dress:  intertrial  interval  effects,  CS  intensity,  stimulus  generalization  and  discrimination. 
These  phenomena  require  a  richer  respresentation  of  CS  input  to  learning  networks  than 
have  been  considered  by  either  model  too  date.  Desmond  (1988)  has  developed  one  ap¬ 
proach  which  can  potentially  address  these  topics.  It  represents  CSs  as  planar  arrays  of 
elements  through  which  activation  spreads  and  decays  in  an  orderly,  yet  stochastic  manner. 
The  planar  array  approach  follows  directly  from  the  foundations  provided  by  the  SBD  and 
VET  models,  and  we  might  anticipate  as  abundant  a  harvest  of  interesting  implications  as 
have  sprung  from  its  forerunners. 
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Introduction 


Previous  investigations  of  the  rabbit  nictitating  membrane  (NM)  response  have  impli¬ 
cated  the  red  nucleus  (RN)  in  the  control  of  the  conditioned  response  (CR).  Lesions  of 
RN  or  the  rubrobulbar  tract  produce  deficits  in  contralateral,  but  not  ipsilateral  CRs  (Ha¬ 
ley,  Lavond  and  Thompson,  1983;  Rosenfield  and  Moore,  1983;  Rosenfield  and  Moore,  1985; 
Rosenfield,  Dovydaitis,  and  Moore,  1985).  HRP  administration  in  the  region  of  the  accessory 
abducens  nucleus  (AAN),  the  nucleus  primarily  responsible  for  defensive  eyeball  retraction, 
results  in  labeling  of  contralateral  RN  neurons  (Desmond,  Rosenfield,  and  Moore,  1983).  In 
cats,  stimulation  of  RN  produces  EPSPs  in  contralateral  AAN  neurons  at  monosynaptic  la¬ 
tencies  (Grant  and  Horcholle-Bossavit,  1986).  In  light  of  evidence  that  cerebellar  deep  nuclei 
(Clark,  McCormick,  Lavond,  and  Thompson,  1984;  Yeo,  Hardiman,  and  Glickstein,  1985a), 
cerebellar  cortex  (Berthier  and  Moore,  1986;  Yeo,  Hardiman,  and  Glickstein,  1985b,  but 
see  Lavond,  Steinmetz,  Yokaitis,  Lee,  and  Thompson,  1986)  and  inferior  olive  (McCormick, 
Steinmetz,  and  Thompson,  1985;  Yeo,  Hardiman,  and  Glickstein,  1986)  are  involved  in  CR 
control,  and  given  that  RN  receives  major  input  from  cerebellar  deep  nuclei,  it  appears  likely 
that  RN  is  one  component  of  a  cerebellar-brain  stem  circuit.  The  purpose  of  this  exper¬ 
iment  was  to  record  the  activity  of  single  RN  neurons  from  awake  rabbits  during  classical 
conditioning  training  (of  the  contralateral  eye)  and  observe  whether  CR-related  unit  activity 
was  elicited. 


Methods 

The  training  procedures  were  similar  to  those  described  by  Desmond  and  Moore  for 
recording  in  the  dorsolateral  pontine  tegmentum  (Desmond  and  Moore,  1986).  We  used  a 
differential  conditioning  procedure  in  order  to  obtain  trials  with  and  without  CRs.  CS-f  and 
CS-  were  1200  or  600  Hz  tones  (counterbalanced)  of  350  ms  duration  and  75  dB  intensity 
presented  in  a  pseudorandom  sequence.  White  noise  of  65  dB  intensity  was  on  continually. 
The  US,  an  electrostimulation  of  .05-0.5  ms  duration  and  1  mA  intensity  delivered  to  the 
periocular  region  of  the  right  eye,  coterminated  with  the  CS+.  The  intertrial  interval  was 
20  s.  The  rabbits  were  trained  for  2-3  days  (100  trials/day)  and  then  surgically  prepared  for 
unit  recordings.  The  rabbits  were  anesthetized  with  a  ketamine  (40  mg/kg)  -  acepromazine 
(1  mg/kg)  mixture  (i.m.).  A  small  recording  hole  was  drilled  on  the  left  side  of  the  skull, 
and  3  additional  holes  were  drilled  for  anchor  screws.  A  recording  chamber  was  cemented  in 
place  and  the  rabbit  was  allowed  to  recover  for  2  days. 

For  unit  recordings,  the  head  of  the  rabbit  was  held  motionless  by  fastening  the  recording 
chamber  to  a  support  bar.  A  tungsten  microelectrode  of  1-5  Mfi  impedance  was  lowered  into 
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the  brain  stem  during  presentation  of  CS+  and  CS-  trials.  Unit  activity  and  NM  movement 
were  taped  for  offline  analysis.  Recording  sites  were  marked  at  the  end  of  the  session  by 
passing  cathodal  current  (20  /rA  for  10  s)  through  the  recording  electrode.  A  maximum  of 
3  recording  tracks  were  made  for  each  rabbit.  The  rabbits  were  then  sacrificed  with  sodium 
pentobarbital  and  transcardially  perfused  for  subsequent  histological  identification  of  the 
recording  tracks. 

For  data  analysis,  tape  recorded  neuronal  activity  and  NM  responses  were  digitized  using 
an  Apple  lie  microcomputer.  Spikes  were  window  discriminated;  the  time  of  occurrence  of 
each  spike,  relative  to  the  trial  onset,  was  recorded  to  the  nearest  250  fis.  The  NM  response 
was  sampled  at  200  Hz.  The  data  were  then  transferred  to  a  Sun  3/50  workstation  for 
subsequent  analysis. 


Results 

A  total  of  195  cells  were  recorded  from  24  New  Zealand  albino  rabbits.  The  137  cells 
judged  to  be  located  in  the  appropriate  region  were  grouped  as  follows:  45  cells  exhibited 
unit  activity  that  was  strongly  correlated  with  the  CR,  26  displayed  moderately  correlated 
activity,  and  66  showed  little  or  no  CR-correlated  activity.  The  45  cells  showing  strong 
CR-correlated  activity  could  be  grouped  into  three  major  types. 


Type  I  (N  =  22).  These  cells  exhibited  an  increase  in  firing  rate  during  the  execution 
of  the  CR.  The  onset  time  of  the  increase  was  correlated  with  the  onset  time  of  the  CR 
(mean  r  =  0.67,  S.D.  =  0.18.)  The  measure  A2,  as  recommended  by  Commenges  and  Seal 
(Commenges  and  Seal,  19b6)  was  <  1  for  each  cell.  These  cells  could  be  further  subdivided 
into  two  groups: 


1.  Cells  with  very  low  baseline  firing  (mean  =  2.52  spikes/s,  S.D.  =  2.45,  N  =  11). 

2.  Cells  that  were  spontaneously  active  (mean  =  30.82  spikes/s,  S.D.  =  12.38,  N  =  11). 

Neuronal  firing  patterns  for  2  Type  I  cells  are  depicted  in  Figure  1  (one  cell/row).  For 
each  cell  there  are  two  graphs.  The  left-side  graph,  which  will  be  referred  to  as  a  peri-stimulus 
plot,  depicts  from  top  to  bottom,  NM  responses  in  order  of  onset  latency,  spike  raster  plots, 
and  spike  counts  collected  in  5  ms  bins.  Vertical  lines  indicate  the  onset  time  of  the  CS  and 
the  US  (on  nonreinforced  trials  the  US  line  indicates  CS  offset).  Spike  activity  prior  to  CS 
onset  represents  baseline  firing.  The  label  in  the  top  left  corner  indicates  the  type  of  CS 
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(600  or  1200  Hz)  and  whether  or  not  a  US  was  present  (+  or  -).  A  label  of  CR  indicates 
that  CR  trial  types  from  both  CS+  and  CS-  were  pooled.  The  right-side  graph,  which  will 
be  referred  to  as  a  peri-event  plot,  depicts  spike  and  NM  responses  for  CR  trials  only;  NM 
responses  and  spike  activity  are  shifted  so  that  CR  onset  times  are  aligned.  Calibration 
marker  represents  100  ms  (X-axis)  and  100  spikes/s  (Y-axis)  for  ail  graphs. 
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Figure  1:  Two  examples  of  Type  I  cells.  For  each  cell,  neuronal  activity  and  behavioral  responses 
are  depicted  in  a  peristimulus  time  histogram  (left  side)  and  a  peri-event  histogram  (right  side). 


Type  II  (N  =  15).  These  cells  also  exhibited  an  increase  in  firing  rate  during  CR  exe¬ 
cution,  but  the  onset  of  the  neural  burst  was  not  correlated  with  the  onset  time  of  the  CR 
(mean  r  =  0.13,  S.D.  =  0.21,  A2  >  1).  Examples  of  firing  patterns  for  Type  II  cells  are 
given  in  Figure  2.  Typically,  these  cells  increased  firing  shortly  after  CS  onset  and  increased 
further  during  the  CR.  These  cells  could  also  be  divided  into  two  groups: 
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1.  Cells  with  very  low  baseline  firing  (mean  =  2.12  spikes/s,  S.D.  =  1.85,  N  =  9). 

2.  Cells  that  were  spontaneously  active  (mean  =  25.55  spikes/s,  S.D.  =  14.66,  N  =  6). 


Figure  2:  Two  examples  of  Type  II  cells. 


Type  III  (N  =  8).  These  cells  exhibited  CR-related  decreases  in  firing  rate.  The  mean 
baseline  firing  rate  for  these  cells  was  23.83  spikes/s,  S.D.  =  10.12.  Examples  of  two  Type 
III  cells  are  given  in  Figure  3. 

A  total  of  14  cells  displayed  both  increases  and  decreases  in  firing  after  the  US  occurred. 
Many  of  these  cells  were  located  in  or  in  close  proximity  to  the  magnocellular  division. 
Examples  of  US  responses  in  2  of  these  cells  are  illustrated  in  Figure  4. 
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Figure  3:  Two  examples  of  Type  III  cells. 

To  determine  the  temporal  relationship  between  neural  activity  and  the  behavior,  we 
inspected  peri-event  plots,  which  were  described  above,  and  graphs  showing  the  cumulative 
sum  of  spike  counts  adjusted  for  baseline  firing  (Ellaway,  1977;  Gibson,  Houk,  and  Kohler- 
man,  1985).  The  latter  graphs  were  constructed  using  the  algorithm,  y(t)  =  £{=1[®(i)  —  *], 
where  y(t )  is  the  cumulative  sum  at  time  bin  t  (10  ms  bins  were  used),  x(i)  is  the  number 
of  spikes  in  time  bin  i,  and  x  is  the  mean  number  of  spikes  per  bin  computed  from  the 
pre-CS  period.  In  these  graphs,  changes  in  firing  rate  are  depicted  as  changes  in  slope  of 
the  line,  where  0  slope  denotes  baseline  firing,  For  Type  II  cells  we  tried  to  differentiate  the 
initial  neural  burst  at  CS  onset  (which  occurred  for  both  CR  and  non-CR  trials)  from  a  later 
CR-specific  discharge.  The  latter  discharge  was  identified  by  a  second  change  in  slope  in  the 
graph;  neuronal  lead  time  was  measured  from  this  point. 
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Figure  4:  Examples  of  complex  neuronal  responses  to  the  unconditioned  stimulus. 

The  results  of  these  temporal  analyses  are  depicted  in  Table  1,  which  shows  the  average 
time  in  ms  by  which  change  in  neuronal  firing  preceded  the  onset  time  of  the  CR  for  each 
cell  type.  “Quiet”  and  “Active”  refer  to  baseline  firing  rates. 

Recording  locations  for  all  cells  in  this  experiment  are  depicted  in  coronal  sections  in 
Figure  5.  Numbers  in  the  bottom  right  corner  of  each  panel  indicate  the  distance  in  mm 
rostral  to  the  most  posterior  pole  of  the  magnocellular  RN.  The  region  where  most  magno- 
cellular  RN  cells  are  found  is  indicated  in  each  section  (see  mRN  label  in  0.50  section).  This 
region  is  depicted  with  a  dotted  line  at  the  2.5  mm  section  because  the  large  cells  are  more 
diffusely  located  at  this  level.  The  parvicellular  division  is  difficult  to  precisely  delimit,  but 


Cell  Type 

Mean 

S.D. 

N 

Type  I  Quiet 

60.5 

37.9 

11 

Type  I  Active 

70.9 

33.6 

11 

Type  II  Quiet 

52.8 

34.5 

9 

Type  II  Active 

58.3 

34.9 

6 

Type  III 

30.6 

60.2 

8 

Table  1:  Time  by  which  changes  in  neuronal  discharge  preceded  the  onset  of  the  conditioned 
response  for  each  cell  type.  Mean  and  standard  deviation  of  the  lead  times  (in  ms)  as  well  as 
sample  size  are  reported. 
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it  extends  both  laterally  and  dorsally  from  the  magnocellular  border  (Gerhard,  1968). 


Figure  5:  Anatomical  locations  of  all  recording  sites  in  this  study.  Numbers  at  the  bottom  right 
of  each  section  represents  the  distance  in  mm  rostral  to  the  posterior  pole  of  the  magnocellular  RN. 

Locations  of  strongly  CR-correlated  Type  I,  Type  II,  and  Type  III  cells  are  illustrated 
in  Figures  6,  7,  and  8,  respectively.  These  figures  show  that  many,  but  not  all,  Type  I  cells 
tend  to  be  found  near  the  dorsal  border  of  the  caudal  magnocellular  RN.  However,  there  is 
considerable  overlap  in  the  anatomical  distributions  of  the  three  cell  types. 


Conclusions 

We  have  found  that  CR-related  changes  in  neuronal  activity  in  RN  is  manifested  primarily 
as  increases  in  firing  rate  that  precede  the  behavior  by  approximately  50-70  ms.  Cells  that 
fired  in  this  manner  were  divided  into  two  types.  Type  I  cells  appeared  to  be  highly  movement 
related,  and  consistently  fired  prior  to  the  onset  of  the  CR.  Type  II  cells  were  not  as  well 
correlated  with  movement.  Many  of  these  cells  increased  firing  shortly  after  CS  onset,  and 
thus,  the  activity  of  these  cells  appeared  to  have  a  sensory  component.  However,  comparisons 
of  neural  activity  on  CR  versus  non-CR  trials  for  these  cells  suggests  that  their  activity  also 
has  a  movement  component.  CR-related  decreases  in  firing  (Type  III  cells)  were  less  often 
encountered,  and  usually  followed  the  onset  of  the  behavior. 

Based  upon  the  sample  gathered  thus  far,  Type  I  cells  with  active  baseline  firing  may 
be  more  closely  associated  with  the  magnocellular  division  of  RN  than  Type  II  or  Type  III 
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Figure  6:  Locations  of  all  stongly  CR-correlated  cells  that  were  classified  as  Type  I.  Cells  that 
exhibited  quiet  (  +  )  and  spontaneously  active  (*)  baseline  firing  are  depicted.  Changes  in  neuronal 
activity  preceded  CR  onset  in  all  but  one  of  these  cells. 

cells.  Most  of  the  recordings  that  were  in  the  magnocellular  division  tended  to  be  located 
at  caudal  levels  of  the  nucleus  and  at  the  dorsal  and  lateral  edges.  Type  I  and  Type  II  cells 
having  low  baseline  firing  rates  tended  to  be  located  dorsal  to  cells  that  were  spontaneously 
active.  These  quiet  cells  were  presumed  to  be  in  the  parvicellular  division  of  RN  as  delimited 
in  Gerhard’s  atlas  (Gerhard,  1968). 

The  results  of  this  experiment  are  consistent  with  RN  involvement  in  CR  control.  Based 
upon  physiological  and  anatomical  evidence  in  cats,  descending  projections  from  RN  could 
influence  contralateral  accessory  abducens  motoneurons  either  directly  (Grant  and  Horcholle- 
Bossavit,  1986)  or  via  trigeminal  relay  (Edwaids,  1972;  Robinson,  Houk,  and  Gibson,  1987). 
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Figure  7s  Locations  of  all  stongly  CR-correlated  cells  that  were  classified  as  Type  II.  Cells  that 
exhibited  quiet  (+)  and  spontaneously  active  (*)  baseline  firing  are  depicted.  Changes  in  neuronal 
activity  preceded  CR  onset  in  all  but  one  of  these  cells. 


Figure  8s  Locations  of  all  stongly  CR-correlated  cells  that  were  classified  as  Type  III.  Decreases 
in  firing  preceded  CR  onset  in  only  3  cases  (*).  The  remaining  5  cells  exhibited  decreases  in  firing 
after  CR  onset  (+). 
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